PRICING
PRODUCT
SOLUTIONS
by use cases
AI Lead ManagementInvoicingSocial MediaProject ManagementData Managementby Industry
learn more
BlogTemplatesVideosYoutubeRESOURCES
COMMUNITIES AND SOCIAL MEDIA
PARTNERS
Debugging Puppeteer scripts can feel overwhelming, but mastering a few key techniques can save you time and frustration. Here's a quick rundown of what you'll learn:
slowMo
mode and headful browser mode to watch your script in action.try-catch
blocks and automate error screenshots for better troubleshooting.const browser = await puppeteer.launch({
headless: false,
slowMo: 100,
devtools: true
});
DEBUG="puppeteer:*"
for detailed logs.By combining these techniques, you'll streamline your debugging process and improve your Puppeteer scripts' reliability.
Set up Puppeteer for debugging with these launch options:
const browser = await puppeteer.launch({
headless: false,
slowMo: 20,
devtools: true
});
To enable detailed logging, run your script with the following command:
DEBUG="puppeteer:*" node script.js
Once configured, use debugging tools to analyze and refine your script's behavior.
Here are some tools to help you resolve issues effectively:
Tool | Purpose | Key Feature |
---|---|---|
Chrome DevTools | Inspect scripts | Console and network tools |
VS Code Debugger | Manage breakpoints | Step-by-step execution |
Screenshot Utility | Visual troubleshooting | Capture page states |
You can also add checkpoints in your code for better visibility:
await page.screenshot({ path: 'before_click.png' });
await page.click('#button');
await page.screenshot({ path: 'after_click.png' });
Good organization is just as important as the tools you use. Break your script into logical modules and include error handling for smoother debugging:
try {
await page.waitForSelector('#target-element');
await page.click('#target-element');
} catch (error) {
console.error(`Navigation failed: ${error.message}`);
await page.screenshot({ path: 'error-state.png' });
}
To reduce the chance of bot detection, integrate the stealth plugin:
const puppeteer = require('puppeteer-extra');
const StealthPlugin = require('puppeteer-extra-plugin-stealth');
puppeteer.use(StealthPlugin());
Finally, ensure proper resource management by implementing cleanup procedures:
async function cleanup() {
if (page) await page.close();
if (browser) await browser.close();
}
This setup provides a solid foundation for visual and console debugging in your projects.
Watching your scripts in action can reveal problems that traditional code analysis might miss. These methods expand your debugging options beyond console logs and error tracking.
SlowMo introduces a delay between Puppeteer actions, making it easier to follow what's happening:
const browser = await puppeteer.launch({
headless: false,
slowMo: 250,
devtools: true
});
The slowMo
value (in milliseconds) controls the delay between actions. Adjust it based on what you're testing:
Operation Type | Recommended SlowMo (ms) | Use Case |
---|---|---|
Simple clicks | 100–250 | Basic navigation steps |
Form filling | 250–500 | Testing input validation |
Dynamic content | 500–1000 | Checking loading states |
Once you've set up SlowMo, pair it with Browser View Mode to monitor how the UI behaves during script execution.
Browser View Mode lets you see your script run in a visible browser window, which is especially helpful for debugging dynamic content and complex interactions.
const browser = await puppeteer.launch({
headless: false,
defaultViewport: { width: 1700, height: 800 },
args: ['--start-maximized']
});
For instance, Acme Corp's QA team used this mode in June 2024 to troubleshoot a web scraping script. They spotted incorrect selectors and fixed them, cutting debugging time by 40%.
To complement this, capture screenshots of important visual states for further analysis.
Screenshots and videos can create a clear record of your script's execution, making debugging easier:
// Screenshot of a specific element
await page.screenshot({
path: 'element-state.png',
clip: {
x: 0,
y: 0,
width: 500,
height: 300
}
});
// Full-page screenshot
await page.screenshot({
path: 'full-page.png',
fullPage: true
});
Start by enabling Browser View Mode, use SlowMo for detailed tracking, and document key moments with screenshots. Together, these steps create a thorough visual debugging process.
Console methods provide a straightforward way to gain text-based insights into how your scripts are behaving. These outputs work alongside visual debugging to give you precise details about script execution.
Puppeteer makes it easy to capture browser messages with event handlers like these:
page.on('console', msg => {
console.log('PAGE LOG:', msg.text());
});
page.on('pageerror', err => {
console.error('PAGE ERROR:', err.message);
});
page.on('requestfailed', request => {
console.error('REQUEST FAILED:', request.url());
});
This setup creates a logging system that tracks console messages, page errors, and failed requests. To make things clearer, you can categorize messages by type:
Message Type | Purpose | Example Output |
---|---|---|
Log | General information | Standard execution flow |
Error | Major issues | Failed operations |
Warning | Potential concerns | Performance slowdowns |
Info | Status updates | Task completion |
Using console.log
wisely can make debugging much easier. Place logs strategically to track progress and identify issues:
// Log before attempting to find an element
console.log(`Looking for element: ${selector}`);
const element = await page.$(selector);
// Log after confirming the element exists
console.log(`Element found: ${!!element}`);
// Log form data before filling it out
console.log(`Form data: ${JSON.stringify(formData)}`);
await page.type('#email', formData.email);
For more complex issues, advanced logging techniques can be a game-changer:
// Enable detailed debugging for Puppeteer
process.env.DEBUG = 'puppeteer:*';
process.env.DEBUG_MAX_STRING_LENGTH = null;
// Monitor pending protocol calls
const browser = await puppeteer.launch({
dumpio: true
});
console.log(browser.debugInfo.pendingProtocolErrors);
One team saw a 40% drop in test failures after adopting detailed protocol logging.
// Filter out specific network domain messages
// Command: DEBUG="puppeteer:*" DEBUG_COLORS=true node script.js 2>&1 | grep -v '"Network'
These methods add a text-based layer to your debugging process, helping you catch and resolve issues more effectively.
Debugging complex Puppeteer scripts involves using effective error-handling strategies and advanced techniques to ensure scripts run smoothly.
Use try-catch blocks to manage errors effectively and keep your script running:
async function navigateAndScreenshot(url, selector) {
try {
await page.goto(url, { waitUntil: 'networkidle0' });
const element = await page.waitForSelector(selector, { timeout: 5000 });
await element.screenshot({ path: 'element.png' });
} catch (error) {
if (error instanceof TimeoutError) {
console.error(`Element ${selector} not found within timeout`);
// Add recovery logic if needed
await page.reload();
} else {
console.error(`Navigation failed: ${error.message}`);
throw error; // Re-throw unexpected errors
}
}
}
You can enhance error handling by combining try-catch blocks with custom error classes for better categorization and response.
Creating custom error classes helps you pinpoint and classify issues more efficiently:
class PuppeteerScriptError extends Error {
constructor(message, details = {}) {
super(message);
this.name = 'PuppeteerScriptError';
this.details = details;
this.timestamp = new Date().toISOString();
}
}
class SelectorError extends PuppeteerScriptError {
constructor(selector, context) {
super(`Failed to find selector: ${selector}`, {
selector,
context,
type: 'SELECTOR_ERROR'
});
this.name = 'SelectorError';
}
}
These classes allow you to track and debug asynchronous operations with more clarity.
Asynchronous code often introduces timing issues and unresolved promises. Tackle these problems with the following techniques:
// Enable detailed debugging for protocol calls
const browser = await puppeteer.launch({
dumpio: true
});
// Monitor unresolved promises periodically
setInterval(() => {
const pending = browser.debugInfo.pendingProtocolErrors;
if (pending.length > 0) {
console.log('Pending protocol calls:', pending);
}
}, 5000);
// Handle async errors with a timeout mechanism
async function safeExecute(promiseFn) {
try {
return await Promise.race([
promiseFn(),
new Promise((_, reject) =>
setTimeout(() => reject(new Error('Operation timed out')), 30000)
)
]);
} catch (error) {
console.error(`Operation failed: ${error.message}`);
throw new PuppeteerScriptError('Execution timeout', {
originalError: error,
operation: promiseFn.name
});
}
}
By using the debugInfo
interface, you can monitor pending callbacks and identify unresolved promises during browser protocol communication.
Debug Level | Purpose | Implementation |
---|---|---|
Basic | Handle common errors | Standard try-catch blocks |
Intermediate | Classify errors | Custom error class hierarchy |
Advanced | Track protocol issues | Debug interface monitoring |
This section dives into frequent challenges with Puppeteer and provides clear fixes to keep your automation scripts running smoothly.
Selector problems can often disrupt script execution. Here's how to handle them effectively:
async function findElement(page) {
try {
const element = await page.waitForSelector('[data-testid="target"]', {
timeout: 5000
});
return element;
} catch {
return page.waitForSelector('.target-class', {
timeout: 5000
});
}
}
For elements inside iframes or Shadow DOM, use these approaches:
// Access iframe content
const frame = await page.frames().find(f => f.name() === 'content-frame');
const button = await frame.$('button[data-hook="create"]');
// Handle Shadow DOM elements
await page.evaluateHandle(selector => {
const element = document.querySelector('parent-element')
.shadowRoot
.querySelector(selector);
return element;
}, 'target-selector');
Properly handling selectors ensures your scripts locate elements reliably.
Once selectors are stable, managing timing is crucial for smooth execution:
await page.setDefaultNavigationTimeout(30000);
await page.setDefaultTimeout(10000);
async function waitForContent(page) {
await Promise.all([
page.waitForNavigation({ waitUntil: 'networkidle0' }),
page.click('#load-more-button')
]);
}
Here’s a quick reference for timing controls:
Timing Issue | Solution | Implementation |
---|---|---|
Page Load | waitForNavigation | Wait for network idle |
Dynamic Content | waitForSelector | Use with appropriate timeout |
AJAX Updates | waitForResponse | Monitor specific network requests |
These strategies help align your script's timing with page behavior.
Even with solid selector and timing strategies, browser crashes can still occur. Here’s how to minimize and recover from them:
const browser = await puppeteer.launch({
args: [
'--disable-dev-shm-usage',
'--enable-gpu',
'--no-first-run',
'--disable-extensions'
]
});
For crash recovery:
let browser;
try {
browser = await puppeteer.launch();
const page = await browser.newPage();
page.on('error', err => {
console.error('Page crashed:', err);
});
await page.goto('https://example.com');
} catch (error) {
console.error('Browser error:', error);
} finally {
if (browser) {
await browser.close();
}
}
If you're working on Linux, check for missing dependencies:
ldd chrome | grep not
To optimize resource usage, adjust browser flags:
const browser = await puppeteer.launch({
args: [
'--disable-dev-shm-usage',
'--disable-accelerated-2d-canvas',
'--disable-gpu'
]
});
Set up automatic recovery for added resilience:
async function checkAndRecoverPage(page) {
if (!page.isClosed()) {
try {
await page.reload();
} catch {
page = await browser.newPage();
}
}
return page;
}
Enhance your scripts for easier maintenance and quicker error resolution by building on proven debugging techniques.
Keep your code readable by grouping configurations and using clear, descriptive names:
// Group related configurations
const browserConfig = {
headless: false,
defaultViewport: { width: 1920, height: 1080 },
args: ['--no-sandbox', '--disable-setuid-sandbox']
};
// Use descriptive function names
async function validatePageContent(page) {
const pageTitle = await page.title();
console.log(`Validating content for page: ${pageTitle}`);
const contentExists = await page.evaluate(() => {
const mainContent = document.querySelector('.main-content');
return {
hasHeader: !!document.querySelector('header'),
hasContent: !!mainContent,
contentLength: mainContent?.textContent.length || 0
};
});
return contentExists;
}
Divide your scripts into separate modules to simplify debugging. This approach isolates selectors, actions, and validations, making it easier to locate and fix errors.
// selectors.js
export const SELECTORS = {
loginForm: '#login-form',
submitButton: '[data-testid="submit-btn"]',
errorMessage: '.error-notification'
};
// actions.js
export async function performLogin(page, credentials) {
await page.type(SELECTORS.loginForm + ' input[name="username"]', credentials.username);
await page.type(SELECTORS.loginForm + ' input[name="password"]', credentials.password);
await Promise.all([
page.waitForNavigation(),
page.click(SELECTORS.submitButton)
]);
}
// validators.js
export async function checkLoginStatus(page) {
const errorElement = await page.$(SELECTORS.errorMessage);
if (errorElement) {
throw new Error('Login failed: ' + await page.evaluate(el => el.textContent, errorElement));
}
}
This modular structure not only organizes your code but also helps streamline error tracking.
Set up error tracking to identify issues quickly and provide detailed context for debugging:
class PuppeteerError extends Error {
constructor(message, action, selector) {
super(message);
this.name = 'PuppeteerError';
this.action = action;
this.selector = selector;
this.timestamp = new Date().toISOString();
}
}
async function executeWithTracking(page, action, description) {
try {
await action();
} catch (error) {
const screenshot = await page.screenshot({
path: `error-${Date.now()}.png`,
fullPage: true
});
throw new PuppeteerError(
`Failed to ${description}`,
error.message,
error.selector
);
}
}
You can also automate logging of console errors and warnings:
page.on('console', message => {
const type = message.type();
const text = message.text();
if (type === 'error' || type === 'warning') {
console.log(`[${type.toUpperCase()}] ${text}`);
// Log to external service or file
logger.log({
level: type,
message: text,
timestamp: new Date().toISOString(),
url: page.url()
});
}
});
Add validation checks to ensure critical operations complete successfully:
async function validateOperation(page, action) {
const beforeState = await page.evaluate(() => ({
url: window.location.href,
elements: document.querySelectorAll('*').length
}));
await action();
const afterState = await page.evaluate(() => ({
url: window.location.href,
elements: document.querySelectorAll('*').length
}));
return {
urlChanged: beforeState.url !== afterState.url,
elementsDelta: afterState.elements - beforeState.elements
};
}
These techniques, combined with earlier debugging methods, help you quickly identify and resolve issues while keeping your scripts maintainable.
Using visual debugging in headful mode with slowMo
allows for immediate feedback on scripts and precise timing adjustments. For more detailed scenarios, the DevTools protocol provides step-by-step debugging and access to process logs for deeper insights.
const browser = await puppeteer.launch({
headless: false,
slowMo: 100,
devtools: true,
dumpio: true
});
To improve your workflow, consider incorporating continuous monitoring and resource management practices alongside these debugging methods.
Now that you have a solid foundation in debugging techniques, here’s how you can optimize and maintain your Puppeteer scripts:
puppeteer-extra-plugin-stealth
plugin to minimize automation detection and reduce script failures.
Here’s an example of a cleanup function to manage resources effectively:
async function cleanupResources(page) {
await page.evaluate(() => {
if (window.performance.memory) {
console.log(`Heap size limit: ${(window.performance.memory.jsHeapSizeLimit / 1024 / 1024).toFixed(2)} MB`);
}
});
await page.close();
}
Stay ahead by regularly checking the Puppeteer GitHub repository for updates, new features, and best practices. Keeping your toolkit current ensures your scripts remain efficient and adaptable as web technologies change.