Debugging Puppeteer Scripts: From slowMo Mode to Advanced Techniques

Debugging Puppeteer scripts can feel overwhelming, but mastering a few key techniques can save you time and frustration. Here's a quick rundown of what you'll learn:

• Start with Visual Debugging: Use slowMo mode and headful browser mode to watch your script in action.
• Capture Errors: Add try-catch blocks and automate error screenshots for better troubleshooting.
• Leverage Console Logs: Track page errors, failed requests, and custom messages for deeper insights.
• Handle Selectors and Timing: Use robust selector strategies and manage timeouts to avoid common pitfalls.
• Organize Your Code: Break scripts into modules for actions, selectors, and validations to simplify maintenance.

Quick Setup Example:

<span class="hljs-keyword">const</span> browser = <span class="hljs-keyword">await</span> puppeteer.<span class="hljs-title function_">launch</span>({
    <span class="hljs-attr">headless</span>: <span class="hljs-literal">false</span>,
    <span class="hljs-attr">slowMo</span>: <span class="hljs-number">100</span>,
    <span class="hljs-attr">devtools</span>: <span class="hljs-literal">true</span>
});

Debugging Tips:

1. Enable Logging: Run with DEBUG="puppeteer:*" for detailed logs.
2. Use Screenshots: Capture page states to see what went wrong.
3. Manage Resources: Always clean up browser and page instances to prevent crashes.

By combining these techniques, you'll streamline your debugging process and improve your Puppeteer scripts' reliability.

Debug Environment Setup

Basic Debug Settings

Set up Puppeteer for debugging with these launch options:

<span class="hljs-keyword">const</span> browser = <span class="hljs-keyword">await</span> puppeteer.<span class="hljs-title function_">launch</span>({
    <span class="hljs-attr">headless</span>: <span class="hljs-literal">false</span>,
    <span class="hljs-attr">slowMo</span>: <span class="hljs-number">20</span>,
    <span class="hljs-attr">devtools</span>: <span class="hljs-literal">true</span>
});

To enable detailed logging, run your script with the following command:

DEBUG=<span class="hljs-string">&quot;puppeteer:*&quot;</span> node script.js

Once configured, use debugging tools to analyze and refine your script's behavior.

Required Debug Tools

Here are some tools to help you resolve issues effectively:

Tool	Purpose	Key Feature
Chrome DevTools	Inspect scripts	Console and network tools
VS Code Debugger	Manage breakpoints	Step-by-step execution
Screenshot Utility	Visual troubleshooting	Capture page states

You can also add checkpoints in your code for better visibility:

<span class="hljs-keyword">await</span> page.<span class="hljs-title function_">screenshot</span>({ <span class="hljs-attr">path</span>: <span class="hljs-string">&#x27;before_click.png&#x27;</span> });
<span class="hljs-keyword">await</span> page.<span class="hljs-title function_">click</span>(<span class="hljs-string">&#x27;#button&#x27;</span>);
<span class="hljs-keyword">await</span> page.<span class="hljs-title function_">screenshot</span>({ <span class="hljs-attr">path</span>: <span class="hljs-string">&#x27;after_click.png&#x27;</span> });

Script Organization

Good organization is just as important as the tools you use. Break your script into logical modules and include error handling for smoother debugging:

<span class="hljs-keyword">try</span> {
    <span class="hljs-keyword">await</span> page.<span class="hljs-title function_">waitForSelector</span>(<span class="hljs-string">&#x27;#target-element&#x27;</span>);
    <span class="hljs-keyword">await</span> page.<span class="hljs-title function_">click</span>(<span class="hljs-string">&#x27;#target-element&#x27;</span>);
} <span class="hljs-keyword">catch</span> (error) {
    <span class="hljs-variable language_">console</span>.<span class="hljs-title function_">error</span>(<span class="hljs-string">`Navigation failed: <span class="hljs-subst">${error.message}</span>`</span>);
    <span class="hljs-keyword">await</span> page.<span class="hljs-title function_">screenshot</span>({ <span class="hljs-attr">path</span>: <span class="hljs-string">&#x27;error-state.png&#x27;</span> });
}

To reduce the chance of bot detection, integrate the stealth plugin:

<span class="hljs-keyword">const</span> puppeteer = <span class="hljs-built_in">require</span>(<span class="hljs-string">&#x27;puppeteer-extra&#x27;</span>);
<span class="hljs-keyword">const</span> <span class="hljs-title class_">StealthPlugin</span> = <span class="hljs-built_in">require</span>(<span class="hljs-string">&#x27;puppeteer-extra-plugin-stealth&#x27;</span>);
puppeteer.<span class="hljs-title function_">use</span>(<span class="hljs-title class_">StealthPlugin</span>());

Finally, ensure proper resource management by implementing cleanup procedures:

<span class="hljs-keyword">async</span> <span class="hljs-keyword">function</span> <span class="hljs-title function_">cleanup</span>(<span class="hljs-params"></span>) {
    <span class="hljs-keyword">if</span> (page) <span class="hljs-keyword">await</span> page.<span class="hljs-title function_">close</span>();
    <span class="hljs-keyword">if</span> (browser) <span class="hljs-keyword">await</span> browser.<span class="hljs-title function_">close</span>();
}

This setup provides a solid foundation for visual and console debugging in your projects.

Puppeteer - 3 Approaches to Consider at Debugging

Visual Debug Methods

Watching your scripts in action can reveal problems that traditional code analysis might miss. These methods expand your debugging options beyond console logs and error tracking.

SlowMo Mode Guide

SlowMo introduces a delay between Puppeteer actions, making it easier to follow what's happening:

<span class="hljs-keyword">const</span> browser = <span class="hljs-keyword">await</span> puppeteer.<span class="hljs-title function_">launch</span>({
    <span class="hljs-attr">headless</span>: <span class="hljs-literal">false</span>,
    <span class="hljs-attr">slowMo</span>: <span class="hljs-number">250</span>,
    <span class="hljs-attr">devtools</span>: <span class="hljs-literal">true</span>
});

The slowMo value (in milliseconds) controls the delay between actions. Adjust it based on what you're testing:

Operation Type	Recommended SlowMo (ms)	Use Case
Simple clicks	100–250	Basic navigation steps
Form filling	250–500	Testing input validation
Dynamic content	500–1000	Checking loading states

Once you've set up SlowMo, pair it with Browser View Mode to monitor how the UI behaves during script execution.

Browser View Mode

Browser View Mode lets you see your script run in a visible browser window, which is especially helpful for debugging dynamic content and complex interactions.

<span class="hljs-keyword">const</span> browser = <span class="hljs-keyword">await</span> puppeteer.<span class="hljs-title function_">launch</span>({
    <span class="hljs-attr">headless</span>: <span class="hljs-literal">false</span>,
    <span class="hljs-attr">defaultViewport</span>: { <span class="hljs-attr">width</span>: <span class="hljs-number">1700</span>, <span class="hljs-attr">height</span>: <span class="hljs-number">800</span> },
    <span class="hljs-attr">args</span>: [<span class="hljs-string">&#x27;--start-maximized&#x27;</span>]
});

For instance, Acme Corp's QA team used this mode in June 2024 to troubleshoot a web scraping script. They spotted incorrect selectors and fixed them, cutting debugging time by 40%.

To complement this, capture screenshots of important visual states for further analysis.

Visual Recording

Screenshots and videos can create a clear record of your script's execution, making debugging easier:

<span class="hljs-comment">// Screenshot of a specific element</span>
<span class="hljs-keyword">await</span> page.<span class="hljs-title function_">screenshot</span>({
    <span class="hljs-attr">path</span>: <span class="hljs-string">&#x27;element-state.png&#x27;</span>,
    <span class="hljs-attr">clip</span>: {
        <span class="hljs-attr">x</span>: <span class="hljs-number">0</span>,
        <span class="hljs-attr">y</span>: <span class="hljs-number">0</span>,
        <span class="hljs-attr">width</span>: <span class="hljs-number">500</span>,
        <span class="hljs-attr">height</span>: <span class="hljs-number">300</span>
    }
});

<span class="hljs-comment">// Full-page screenshot</span>
<span class="hljs-keyword">await</span> page.<span class="hljs-title function_">screenshot</span>({
    <span class="hljs-attr">path</span>: <span class="hljs-string">&#x27;full-page.png&#x27;</span>,
    <span class="hljs-attr">fullPage</span>: <span class="hljs-literal">true</span>
});

Start by enabling Browser View Mode, use SlowMo for detailed tracking, and document key moments with screenshots. Together, these steps create a thorough visual debugging process.

Console Debug Methods

Console methods provide a straightforward way to gain text-based insights into how your scripts are behaving. These outputs work alongside visual debugging to give you precise details about script execution.

Console Message Tracking

Puppeteer makes it easy to capture browser messages with event handlers like these:

page.<span class="hljs-title function_">on</span>(<span class="hljs-string">&#x27;console&#x27;</span>, <span class="hljs-function"><span class="hljs-params">msg</span> =&gt;</span> {
    <span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(<span class="hljs-string">&#x27;PAGE LOG:&#x27;</span>, msg.<span class="hljs-title function_">text</span>());
});

page.<span class="hljs-title function_">on</span>(<span class="hljs-string">&#x27;pageerror&#x27;</span>, <span class="hljs-function"><span class="hljs-params">err</span> =&gt;</span> {
    <span class="hljs-variable language_">console</span>.<span class="hljs-title function_">error</span>(<span class="hljs-string">&#x27;PAGE ERROR:&#x27;</span>, err.<span class="hljs-property">message</span>);
});

page.<span class="hljs-title function_">on</span>(<span class="hljs-string">&#x27;requestfailed&#x27;</span>, <span class="hljs-function"><span class="hljs-params">request</span> =&gt;</span> {
    <span class="hljs-variable language_">console</span>.<span class="hljs-title function_">error</span>(<span class="hljs-string">&#x27;REQUEST FAILED:&#x27;</span>, request.<span class="hljs-title function_">url</span>());
});

This setup creates a logging system that tracks console messages, page errors, and failed requests. To make things clearer, you can categorize messages by type:

Message Type	Purpose	Example Output
Log	General information	Standard execution flow
Error	Major issues	Failed operations
Warning	Potential concerns	Performance slowdowns
Info	Status updates	Task completion

Console.log Best Practices

Using console.log wisely can make debugging much easier. Place logs strategically to track progress and identify issues:

<span class="hljs-comment">// Log before attempting to find an element</span>
<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(<span class="hljs-string">`Looking for element: <span class="hljs-subst">${selector}</span>`</span>);
<span class="hljs-keyword">const</span> element = <span class="hljs-keyword">await</span> page.$(selector);
<span class="hljs-comment">// Log after confirming the element exists</span>
<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(<span class="hljs-string">`Element found: <span class="hljs-subst">${!!element}</span>`</span>);

<span class="hljs-comment">// Log form data before filling it out</span>
<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(<span class="hljs-string">`Form data: <span class="hljs-subst">${<span class="hljs-built_in">JSON</span>.stringify(formData)}</span>`</span>);
<span class="hljs-keyword">await</span> page.<span class="hljs-title function_">type</span>(<span class="hljs-string">&#x27;#email&#x27;</span>, formData.<span class="hljs-property">email</span>);

Extended Logging Methods

For more complex issues, advanced logging techniques can be a game-changer:

<span class="hljs-comment">// Enable detailed debugging for Puppeteer</span>
process.<span class="hljs-property">env</span>.<span class="hljs-property">DEBUG</span> = <span class="hljs-string">&#x27;puppeteer:*&#x27;</span>;
process.<span class="hljs-property">env</span>.<span class="hljs-property">DEBUG_MAX_STRING_LENGTH</span> = <span class="hljs-literal">null</span>;

<span class="hljs-comment">// Monitor pending protocol calls</span>
<span class="hljs-keyword">const</span> browser = <span class="hljs-keyword">await</span> puppeteer.<span class="hljs-title function_">launch</span>({
    <span class="hljs-attr">dumpio</span>: <span class="hljs-literal">true</span>
});
<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(browser.<span class="hljs-property">debugInfo</span>.<span class="hljs-property">pendingProtocolErrors</span>);

One team saw a 40% drop in test failures after adopting detailed protocol logging.

<span class="hljs-comment">// Filter out specific network domain messages</span>
<span class="hljs-comment">// Command: DEBUG=&quot;puppeteer:*&quot; DEBUG_COLORS=true node script.js 2&gt;&amp;1 | grep -v &#x27;&quot;Network&#x27;</span>

These methods add a text-based layer to your debugging process, helping you catch and resolve issues more effectively.

sbb-itb-23997f1

Advanced Debug Methods

Debugging complex Puppeteer scripts involves using effective error-handling strategies and advanced techniques to ensure scripts run smoothly.

Try-Catch for Error Handling

Use try-catch blocks to manage errors effectively and keep your script running:

<span class="hljs-keyword">async</span> <span class="hljs-keyword">function</span> <span class="hljs-title function_">navigateAndScreenshot</span>(<span class="hljs-params">url, selector</span>) {
    <span class="hljs-keyword">try</span> {
        <span class="hljs-keyword">await</span> page.<span class="hljs-title function_">goto</span>(url, { <span class="hljs-attr">waitUntil</span>: <span class="hljs-string">&#x27;networkidle0&#x27;</span> });
        <span class="hljs-keyword">const</span> element = <span class="hljs-keyword">await</span> page.<span class="hljs-title function_">waitForSelector</span>(selector, { <span class="hljs-attr">timeout</span>: <span class="hljs-number">5000</span> });
        <span class="hljs-keyword">await</span> element.<span class="hljs-title function_">screenshot</span>({ <span class="hljs-attr">path</span>: <span class="hljs-string">&#x27;element.png&#x27;</span> });
    } <span class="hljs-keyword">catch</span> (error) {
        <span class="hljs-keyword">if</span> (error <span class="hljs-keyword">instanceof</span> <span class="hljs-title class_">TimeoutError</span>) {
            <span class="hljs-variable language_">console</span>.<span class="hljs-title function_">error</span>(<span class="hljs-string">`Element <span class="hljs-subst">${selector}</span> not found within timeout`</span>);
            <span class="hljs-comment">// Add recovery logic if needed</span>
            <span class="hljs-keyword">await</span> page.<span class="hljs-title function_">reload</span>();
        } <span class="hljs-keyword">else</span> {
            <span class="hljs-variable language_">console</span>.<span class="hljs-title function_">error</span>(<span class="hljs-string">`Navigation failed: <span class="hljs-subst">${error.message}</span>`</span>);
            <span class="hljs-keyword">throw</span> error; <span class="hljs-comment">// Re-throw unexpected errors</span>
        }
    }
}

You can enhance error handling by combining try-catch blocks with custom error classes for better categorization and response.

Custom Error Classes

Creating custom error classes helps you pinpoint and classify issues more efficiently:

<span class="hljs-keyword">class</span> <span class="hljs-title class_">PuppeteerScriptError</span> <span class="hljs-keyword">extends</span> <span class="hljs-title class_ inherited__">Error</span> {
    <span class="hljs-title function_">constructor</span>(<span class="hljs-params">message, details = {}</span>) {
        <span class="hljs-variable language_">super</span>(message);
        <span class="hljs-variable language_">this</span>.<span class="hljs-property">name</span> = <span class="hljs-string">&#x27;PuppeteerScriptError&#x27;</span>;
        <span class="hljs-variable language_">this</span>.<span class="hljs-property">details</span> = details;
        <span class="hljs-variable language_">this</span>.<span class="hljs-property">timestamp</span> = <span class="hljs-keyword">new</span> <span class="hljs-title class_">Date</span>().<span class="hljs-title function_">toISOString</span>();
    }
}

<span class="hljs-keyword">class</span> <span class="hljs-title class_">SelectorError</span> <span class="hljs-keyword">extends</span> <span class="hljs-title class_ inherited__">PuppeteerScriptError</span> {
    <span class="hljs-title function_">constructor</span>(<span class="hljs-params">selector, context</span>) {
        <span class="hljs-variable language_">super</span>(<span class="hljs-string">`Failed to find selector: <span class="hljs-subst">${selector}</span>`</span>, {
            selector,
            context,
            <span class="hljs-attr">type</span>: <span class="hljs-string">&#x27;SELECTOR_ERROR&#x27;</span>
        });
        <span class="hljs-variable language_">this</span>.<span class="hljs-property">name</span> = <span class="hljs-string">&#x27;SelectorError&#x27;</span>;
    }
}

These classes allow you to track and debug asynchronous operations with more clarity.

Debugging Asynchronous Code

Asynchronous code often introduces timing issues and unresolved promises. Tackle these problems with the following techniques:

<span class="hljs-comment">// Enable detailed debugging for protocol calls</span>
<span class="hljs-keyword">const</span> browser = <span class="hljs-keyword">await</span> puppeteer.<span class="hljs-title function_">launch</span>({
    <span class="hljs-attr">dumpio</span>: <span class="hljs-literal">true</span>
});

<span class="hljs-comment">// Monitor unresolved promises periodically</span>
<span class="hljs-built_in">setInterval</span>(<span class="hljs-function">() =&gt;</span> {
    <span class="hljs-keyword">const</span> pending = browser.<span class="hljs-property">debugInfo</span>.<span class="hljs-property">pendingProtocolErrors</span>;
    <span class="hljs-keyword">if</span> (pending.<span class="hljs-property">length</span> &gt; <span class="hljs-number">0</span>) {
        <span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(<span class="hljs-string">&#x27;Pending protocol calls:&#x27;</span>, pending);
    }
}, <span class="hljs-number">5000</span>);

<span class="hljs-comment">// Handle async errors with a timeout mechanism</span>
<span class="hljs-keyword">async</span> <span class="hljs-keyword">function</span> <span class="hljs-title function_">safeExecute</span>(<span class="hljs-params">promiseFn</span>) {
    <span class="hljs-keyword">try</span> {
        <span class="hljs-keyword">return</span> <span class="hljs-keyword">await</span> <span class="hljs-title class_">Promise</span>.<span class="hljs-title function_">race</span>([
            <span class="hljs-title function_">promiseFn</span>(),
            <span class="hljs-keyword">new</span> <span class="hljs-title class_">Promise</span>(<span class="hljs-function">(<span class="hljs-params">_, reject</span>) =&gt;</span> 
                <span class="hljs-built_in">setTimeout</span>(<span class="hljs-function">() =&gt;</span> <span class="hljs-title function_">reject</span>(<span class="hljs-keyword">new</span> <span class="hljs-title class_">Error</span>(<span class="hljs-string">&#x27;Operation timed out&#x27;</span>)), <span class="hljs-number">30000</span>)
            )
        ]);
    } <span class="hljs-keyword">catch</span> (error) {
        <span class="hljs-variable language_">console</span>.<span class="hljs-title function_">error</span>(<span class="hljs-string">`Operation failed: <span class="hljs-subst">${error.message}</span>`</span>);
        <span class="hljs-keyword">throw</span> <span class="hljs-keyword">new</span> <span class="hljs-title class_">PuppeteerScriptError</span>(<span class="hljs-string">&#x27;Execution timeout&#x27;</span>, {
            <span class="hljs-attr">originalError</span>: error,
            <span class="hljs-attr">operation</span>: promiseFn.<span class="hljs-property">name</span>
        });
    }
}

By using the debugInfo interface, you can monitor pending callbacks and identify unresolved promises during browser protocol communication.

Debug Level	Purpose	Implementation
Basic	Handle common errors	Standard try-catch blocks
Intermediate	Classify errors	Custom error class hierarchy
Advanced	Track protocol issues	Debug interface monitoring

Common Issue Solutions

This section dives into frequent challenges with Puppeteer and provides clear fixes to keep your automation scripts running smoothly.

Selector Issues

Selector problems can often disrupt script execution. Here's how to handle them effectively:

<span class="hljs-keyword">async</span> <span class="hljs-keyword">function</span> <span class="hljs-title function_">findElement</span>(<span class="hljs-params">page</span>) {
  <span class="hljs-keyword">try</span> {
    <span class="hljs-keyword">const</span> element = <span class="hljs-keyword">await</span> page.<span class="hljs-title function_">waitForSelector</span>(<span class="hljs-string">&#x27;[data-testid=&quot;target&quot;]&#x27;</span>, {
      <span class="hljs-attr">timeout</span>: <span class="hljs-number">5000</span>
    });
    <span class="hljs-keyword">return</span> element;
  } <span class="hljs-keyword">catch</span> {
    <span class="hljs-keyword">return</span> page.<span class="hljs-title function_">waitForSelector</span>(<span class="hljs-string">&#x27;.target-class&#x27;</span>, {
      <span class="hljs-attr">timeout</span>: <span class="hljs-number">5000</span>
    });
  }
}

For elements inside iframes or Shadow DOM, use these approaches:

<span class="hljs-comment">// Access iframe content</span>
<span class="hljs-keyword">const</span> frame = <span class="hljs-keyword">await</span> page.<span class="hljs-title function_">frames</span>().<span class="hljs-title function_">find</span>(<span class="hljs-function"><span class="hljs-params">f</span> =&gt;</span> f.<span class="hljs-title function_">name</span>() === <span class="hljs-string">&#x27;content-frame&#x27;</span>);
<span class="hljs-keyword">const</span> button = <span class="hljs-keyword">await</span> frame.$(<span class="hljs-string">&#x27;button[data-hook=&quot;create&quot;]&#x27;</span>);

<span class="hljs-comment">// Handle Shadow DOM elements</span>
<span class="hljs-keyword">await</span> page.<span class="hljs-title function_">evaluateHandle</span>(<span class="hljs-function"><span class="hljs-params">selector</span> =&gt;</span> {
  <span class="hljs-keyword">const</span> element = <span class="hljs-variable language_">document</span>.<span class="hljs-title function_">querySelector</span>(<span class="hljs-string">&#x27;parent-element&#x27;</span>)
    .<span class="hljs-property">shadowRoot</span>
    .<span class="hljs-title function_">querySelector</span>(selector);
  <span class="hljs-keyword">return</span> element;
}, <span class="hljs-string">&#x27;target-selector&#x27;</span>);

Properly handling selectors ensures your scripts locate elements reliably.

Timing Problems

Once selectors are stable, managing timing is crucial for smooth execution:

<span class="hljs-keyword">await</span> page.<span class="hljs-title function_">setDefaultNavigationTimeout</span>(<span class="hljs-number">30000</span>);
<span class="hljs-keyword">await</span> page.<span class="hljs-title function_">setDefaultTimeout</span>(<span class="hljs-number">10000</span>);

<span class="hljs-keyword">async</span> <span class="hljs-keyword">function</span> <span class="hljs-title function_">waitForContent</span>(<span class="hljs-params">page</span>) {
  <span class="hljs-keyword">await</span> <span class="hljs-title class_">Promise</span>.<span class="hljs-title function_">all</span>([
    page.<span class="hljs-title function_">waitForNavigation</span>({ <span class="hljs-attr">waitUntil</span>: <span class="hljs-string">&#x27;networkidle0&#x27;</span> }),
    page.<span class="hljs-title function_">click</span>(<span class="hljs-string">&#x27;#load-more-button&#x27;</span>)
  ]);
}

Here’s a quick reference for timing controls:

Timing Issue	Solution	Implementation
Page Load	waitForNavigation	Wait for network idle
Dynamic Content	waitForSelector	Use with appropriate timeout
AJAX Updates	waitForResponse	Monitor specific network requests

These strategies help align your script's timing with page behavior.

Browser Crash Fixes

Even with solid selector and timing strategies, browser crashes can still occur. Here’s how to minimize and recover from them:

<span class="hljs-keyword">const</span> browser = <span class="hljs-keyword">await</span> puppeteer.<span class="hljs-title function_">launch</span>({
  <span class="hljs-attr">args</span>: [
    <span class="hljs-string">&#x27;--disable-dev-shm-usage&#x27;</span>,
    <span class="hljs-string">&#x27;--enable-gpu&#x27;</span>,
    <span class="hljs-string">&#x27;--no-first-run&#x27;</span>,
    <span class="hljs-string">&#x27;--disable-extensions&#x27;</span>
  ]
});

For crash recovery:

<span class="hljs-keyword">let</span> browser;
<span class="hljs-keyword">try</span> {
  browser = <span class="hljs-keyword">await</span> puppeteer.<span class="hljs-title function_">launch</span>();
  <span class="hljs-keyword">const</span> page = <span class="hljs-keyword">await</span> browser.<span class="hljs-title function_">newPage</span>();

  page.<span class="hljs-title function_">on</span>(<span class="hljs-string">&#x27;error&#x27;</span>, <span class="hljs-function"><span class="hljs-params">err</span> =&gt;</span> {
    <span class="hljs-variable language_">console</span>.<span class="hljs-title function_">error</span>(<span class="hljs-string">&#x27;Page crashed:&#x27;</span>, err);
  });

  <span class="hljs-keyword">await</span> page.<span class="hljs-title function_">goto</span>(<span class="hljs-string">&#x27;https://example.com&#x27;</span>);
} <span class="hljs-keyword">catch</span> (error) {
  <span class="hljs-variable language_">console</span>.<span class="hljs-title function_">error</span>(<span class="hljs-string">&#x27;Browser error:&#x27;</span>, error);
} <span class="hljs-keyword">finally</span> {
  <span class="hljs-keyword">if</span> (browser) {
    <span class="hljs-keyword">await</span> browser.<span class="hljs-title function_">close</span>();
  }
}

If you're working on Linux, check for missing dependencies:

ldd chrome | grep not

To optimize resource usage, adjust browser flags:

<span class="hljs-keyword">const</span> browser = <span class="hljs-keyword">await</span> puppeteer.<span class="hljs-title function_">launch</span>({
  <span class="hljs-attr">args</span>: [
    <span class="hljs-string">&#x27;--disable-dev-shm-usage&#x27;</span>,
    <span class="hljs-string">&#x27;--disable-accelerated-2d-canvas&#x27;</span>,
    <span class="hljs-string">&#x27;--disable-gpu&#x27;</span>
  ]
});

Set up automatic recovery for added resilience:

<span class="hljs-keyword">async</span> <span class="hljs-keyword">function</span> <span class="hljs-title function_">checkAndRecoverPage</span>(<span class="hljs-params">page</span>) {
  <span class="hljs-keyword">if</span> (!page.<span class="hljs-title function_">isClosed</span>()) {
    <span class="hljs-keyword">try</span> {
      <span class="hljs-keyword">await</span> page.<span class="hljs-title function_">reload</span>();
    } <span class="hljs-keyword">catch</span> {
      page = <span class="hljs-keyword">await</span> browser.<span class="hljs-title function_">newPage</span>();
    }
  }
  <span class="hljs-keyword">return</span> page;
}

Script Debug Optimization

Enhance your scripts for easier maintenance and quicker error resolution by building on proven debugging techniques.

Code Clarity

Keep your code readable by grouping configurations and using clear, descriptive names:

<span class="hljs-comment">// Group related configurations</span>
<span class="hljs-keyword">const</span> browserConfig = {
  <span class="hljs-attr">headless</span>: <span class="hljs-literal">false</span>,
  <span class="hljs-attr">defaultViewport</span>: { <span class="hljs-attr">width</span>: <span class="hljs-number">1920</span>, <span class="hljs-attr">height</span>: <span class="hljs-number">1080</span> },
  <span class="hljs-attr">args</span>: [<span class="hljs-string">&#x27;--no-sandbox&#x27;</span>, <span class="hljs-string">&#x27;--disable-setuid-sandbox&#x27;</span>]
};

<span class="hljs-comment">// Use descriptive function names</span>
<span class="hljs-keyword">async</span> <span class="hljs-keyword">function</span> <span class="hljs-title function_">validatePageContent</span>(<span class="hljs-params">page</span>) {
  <span class="hljs-keyword">const</span> pageTitle = <span class="hljs-keyword">await</span> page.<span class="hljs-title function_">title</span>();
  <span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(<span class="hljs-string">`Validating content for page: <span class="hljs-subst">${pageTitle}</span>`</span>);

  <span class="hljs-keyword">const</span> contentExists = <span class="hljs-keyword">await</span> page.<span class="hljs-title function_">evaluate</span>(<span class="hljs-function">() =&gt;</span> {
    <span class="hljs-keyword">const</span> mainContent = <span class="hljs-variable language_">document</span>.<span class="hljs-title function_">querySelector</span>(<span class="hljs-string">&#x27;.main-content&#x27;</span>);
    <span class="hljs-keyword">return</span> {
      <span class="hljs-attr">hasHeader</span>: !!<span class="hljs-variable language_">document</span>.<span class="hljs-title function_">querySelector</span>(<span class="hljs-string">&#x27;header&#x27;</span>),
      <span class="hljs-attr">hasContent</span>: !!mainContent,
      <span class="hljs-attr">contentLength</span>: mainContent?.<span class="hljs-property">textContent</span>.<span class="hljs-property">length</span> || <span class="hljs-number">0</span>
    };
  });

  <span class="hljs-keyword">return</span> contentExists;
}

Module Organization

Divide your scripts into separate modules to simplify debugging. This approach isolates selectors, actions, and validations, making it easier to locate and fix errors.

<span class="hljs-comment">// selectors.js</span>
<span class="hljs-keyword">export</span> <span class="hljs-keyword">const</span> <span class="hljs-variable constant_">SELECTORS</span> = {
  <span class="hljs-attr">loginForm</span>: <span class="hljs-string">&#x27;#login-form&#x27;</span>,
  <span class="hljs-attr">submitButton</span>: <span class="hljs-string">&#x27;[data-testid=&quot;submit-btn&quot;]&#x27;</span>,
  <span class="hljs-attr">errorMessage</span>: <span class="hljs-string">&#x27;.error-notification&#x27;</span>
};

<span class="hljs-comment">// actions.js</span>
<span class="hljs-keyword">export</span> <span class="hljs-keyword">async</span> <span class="hljs-keyword">function</span> <span class="hljs-title function_">performLogin</span>(<span class="hljs-params">page, credentials</span>) {
  <span class="hljs-keyword">await</span> page.<span class="hljs-title function_">type</span>(<span class="hljs-variable constant_">SELECTORS</span>.<span class="hljs-property">loginForm</span> + <span class="hljs-string">&#x27; input[name=&quot;username&quot;]&#x27;</span>, credentials.<span class="hljs-property">username</span>);
  <span class="hljs-keyword">await</span> page.<span class="hljs-title function_">type</span>(<span class="hljs-variable constant_">SELECTORS</span>.<span class="hljs-property">loginForm</span> + <span class="hljs-string">&#x27; input[name=&quot;password&quot;]&#x27;</span>, credentials.<span class="hljs-property">password</span>);
  <span class="hljs-keyword">await</span> <span class="hljs-title class_">Promise</span>.<span class="hljs-title function_">all</span>([
    page.<span class="hljs-title function_">waitForNavigation</span>(),
    page.<span class="hljs-title function_">click</span>(<span class="hljs-variable constant_">SELECTORS</span>.<span class="hljs-property">submitButton</span>)
  ]);
}

<span class="hljs-comment">// validators.js</span>
<span class="hljs-keyword">export</span> <span class="hljs-keyword">async</span> <span class="hljs-keyword">function</span> <span class="hljs-title function_">checkLoginStatus</span>(<span class="hljs-params">page</span>) {
  <span class="hljs-keyword">const</span> errorElement = <span class="hljs-keyword">await</span> page.$(<span class="hljs-variable constant_">SELECTORS</span>.<span class="hljs-property">errorMessage</span>);
  <span class="hljs-keyword">if</span> (errorElement) {
    <span class="hljs-keyword">throw</span> <span class="hljs-keyword">new</span> <span class="hljs-title class_">Error</span>(<span class="hljs-string">&#x27;Login failed: &#x27;</span> + <span class="hljs-keyword">await</span> page.<span class="hljs-title function_">evaluate</span>(<span class="hljs-function"><span class="hljs-params">el</span> =&gt;</span> el.<span class="hljs-property">textContent</span>, errorElement));
  }
}

This modular structure not only organizes your code but also helps streamline error tracking.

Error Tracking Setup

Set up error tracking to identify issues quickly and provide detailed context for debugging:

<span class="hljs-keyword">class</span> <span class="hljs-title class_">PuppeteerError</span> <span class="hljs-keyword">extends</span> <span class="hljs-title class_ inherited__">Error</span> {
  <span class="hljs-title function_">constructor</span>(<span class="hljs-params">message, action, selector</span>) {
    <span class="hljs-variable language_">super</span>(message);
    <span class="hljs-variable language_">this</span>.<span class="hljs-property">name</span> = <span class="hljs-string">&#x27;PuppeteerError&#x27;</span>;
    <span class="hljs-variable language_">this</span>.<span class="hljs-property">action</span> = action;
    <span class="hljs-variable language_">this</span>.<span class="hljs-property">selector</span> = selector;
    <span class="hljs-variable language_">this</span>.<span class="hljs-property">timestamp</span> = <span class="hljs-keyword">new</span> <span class="hljs-title class_">Date</span>().<span class="hljs-title function_">toISOString</span>();
  }
}

<span class="hljs-keyword">async</span> <span class="hljs-keyword">function</span> <span class="hljs-title function_">executeWithTracking</span>(<span class="hljs-params">page, action, description</span>) {
  <span class="hljs-keyword">try</span> {
    <span class="hljs-keyword">await</span> <span class="hljs-title function_">action</span>();
  } <span class="hljs-keyword">catch</span> (error) {
    <span class="hljs-keyword">const</span> screenshot = <span class="hljs-keyword">await</span> page.<span class="hljs-title function_">screenshot</span>({
      <span class="hljs-attr">path</span>: <span class="hljs-string">`error-<span class="hljs-subst">${<span class="hljs-built_in">Date</span>.now()}</span>.png`</span>,
      <span class="hljs-attr">fullPage</span>: <span class="hljs-literal">true</span>
    });

    <span class="hljs-keyword">throw</span> <span class="hljs-keyword">new</span> <span class="hljs-title class_">PuppeteerError</span>(
      <span class="hljs-string">`Failed to <span class="hljs-subst">${description}</span>`</span>,
      error.<span class="hljs-property">message</span>,
      error.<span class="hljs-property">selector</span>
    );
  }
}

You can also automate logging of console errors and warnings:

page.<span class="hljs-title function_">on</span>(<span class="hljs-string">&#x27;console&#x27;</span>, <span class="hljs-function"><span class="hljs-params">message</span> =&gt;</span> {
  <span class="hljs-keyword">const</span> type = message.<span class="hljs-title function_">type</span>();
  <span class="hljs-keyword">const</span> text = message.<span class="hljs-title function_">text</span>();

  <span class="hljs-keyword">if</span> (type === <span class="hljs-string">&#x27;error&#x27;</span> || type === <span class="hljs-string">&#x27;warning&#x27;</span>) {
    <span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(<span class="hljs-string">`[<span class="hljs-subst">${type.toUpperCase()}</span>] <span class="hljs-subst">${text}</span>`</span>);

    <span class="hljs-comment">// Log to external service or file</span>
    logger.<span class="hljs-title function_">log</span>({
      <span class="hljs-attr">level</span>: type,
      <span class="hljs-attr">message</span>: text,
      <span class="hljs-attr">timestamp</span>: <span class="hljs-keyword">new</span> <span class="hljs-title class_">Date</span>().<span class="hljs-title function_">toISOString</span>(),
      <span class="hljs-attr">url</span>: page.<span class="hljs-title function_">url</span>()
    });
  }
});

Validation for Critical Operations

Add validation checks to ensure critical operations complete successfully:

<span class="hljs-keyword">async</span> <span class="hljs-keyword">function</span> <span class="hljs-title function_">validateOperation</span>(<span class="hljs-params">page, action</span>) {
  <span class="hljs-keyword">const</span> beforeState = <span class="hljs-keyword">await</span> page.<span class="hljs-title function_">evaluate</span>(<span class="hljs-function">() =&gt;</span> ({
    <span class="hljs-attr">url</span>: <span class="hljs-variable language_">window</span>.<span class="hljs-property">location</span>.<span class="hljs-property">href</span>,
    <span class="hljs-attr">elements</span>: <span class="hljs-variable language_">document</span>.<span class="hljs-title function_">querySelectorAll</span>(<span class="hljs-string">&#x27;*&#x27;</span>).<span class="hljs-property">length</span>
  }));

  <span class="hljs-keyword">await</span> <span class="hljs-title function_">action</span>();

  <span class="hljs-keyword">const</span> afterState = <span class="hljs-keyword">await</span> page.<span class="hljs-title function_">evaluate</span>(<span class="hljs-function">() =&gt;</span> ({
    <span class="hljs-attr">url</span>: <span class="hljs-variable language_">window</span>.<span class="hljs-property">location</span>.<span class="hljs-property">href</span>,
    <span class="hljs-attr">elements</span>: <span class="hljs-variable language_">document</span>.<span class="hljs-title function_">querySelectorAll</span>(<span class="hljs-string">&#x27;*&#x27;</span>).<span class="hljs-property">length</span>
  }));

  <span class="hljs-keyword">return</span> {
    <span class="hljs-attr">urlChanged</span>: beforeState.<span class="hljs-property">url</span> !== afterState.<span class="hljs-property">url</span>,
    <span class="hljs-attr">elementsDelta</span>: afterState.<span class="hljs-property">elements</span> - beforeState.<span class="hljs-property">elements</span>
  };
}

These techniques, combined with earlier debugging methods, help you quickly identify and resolve issues while keeping your scripts maintainable.

Conclusion

Key Debugging Techniques

Using visual debugging in headful mode with slowMo allows for immediate feedback on scripts and precise timing adjustments. For more detailed scenarios, the DevTools protocol provides step-by-step debugging and access to process logs for deeper insights.

<span class="hljs-keyword">const</span> browser = <span class="hljs-keyword">await</span> puppeteer.<span class="hljs-title function_">launch</span>({
    <span class="hljs-attr">headless</span>: <span class="hljs-literal">false</span>,
    <span class="hljs-attr">slowMo</span>: <span class="hljs-number">100</span>,
    <span class="hljs-attr">devtools</span>: <span class="hljs-literal">true</span>,
    <span class="hljs-attr">dumpio</span>: <span class="hljs-literal">true</span>
});

To improve your workflow, consider incorporating continuous monitoring and resource management practices alongside these debugging methods.

Next Steps

Now that you have a solid foundation in debugging techniques, here’s how you can optimize and maintain your Puppeteer scripts:

• Performance Monitoring: Use detailed logging to track execution times and resource usage. This helps pinpoint bottlenecks and makes debugging more efficient.
• Error Prevention: Add the puppeteer-extra-plugin-stealth plugin to minimize automation detection and reduce script failures.
• Resource Management: Focus on efficient memory usage and implement cleanup routines to keep your scripts running smoothly.

Here’s an example of a cleanup function to manage resources effectively:

<span class="hljs-keyword">async</span> <span class="hljs-keyword">function</span> <span class="hljs-title function_">cleanupResources</span>(<span class="hljs-params">page</span>) {
    <span class="hljs-keyword">await</span> page.<span class="hljs-title function_">evaluate</span>(<span class="hljs-function">() =&gt;</span> {
        <span class="hljs-keyword">if</span> (<span class="hljs-variable language_">window</span>.<span class="hljs-property">performance</span>.<span class="hljs-property">memory</span>) {
            <span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(<span class="hljs-string">`Heap size limit: <span class="hljs-subst">${(<span class="hljs-variable language_">window</span>.performance.memory.jsHeapSizeLimit / <span class="hljs-number">1024</span> / <span class="hljs-number">1024</span>).toFixed(<span class="hljs-number">2</span>)}</span> MB`</span>);
        }
    });
    <span class="hljs-keyword">await</span> page.<span class="hljs-title function_">close</span>();
}

Stay ahead by regularly checking the Puppeteer GitHub repository for updates, new features, and best practices. Keeping your toolkit current ensures your scripts remain efficient and adaptable as web technologies change.

Related posts

• Installing and Configuring Puppeteer: Solving Common Dependency and Chromium Issues
• Optimizing Wait Strategies in Puppeteer: A Complete Guide to Waiting Methods
• Browser Automation with Puppeteer and JavaScript: Practical Implementation in Node.js
• Configuring Headless Mode in Puppeteer: Balancing Speed and Functionality