Node JS Nested loop. Iteration not completing

I have an array of website urls. I am looping each with another loop for links form its sitemap.
The code I have works but finishes the inner sitemap loop way too early (after 5 iterations) and goes to the next url.

Modules I am usng are puppeteer, mocha lodash and fs extra.
I attempted lodash in the second loop but received a memory leak so I went for a standard loop.

const {urls} = require('./common')

 describe('Test', async () => {
      describe('url loop',async () => {
         _.forEach(urls, async (url) => {
            it('screenshot', async () => {
                await page.goto(url + '/sitemap')
                let sub_domain = url.replace('https://www.', '')
                let domain = sub_domain.substr(0, sub_domain.indexOf('.co'))
                    return Array.from(document.querySelectorAll('#sitemap a')).map(val => val.href);
                  });

                  let links = await page.evaluate(() => {
                      return Array.from(document.querySelectorAll('#sitemap a')).map(val => val.href);
                      });


                    for(x=0; x<links.length; x++){
                      console.log(x)
                      await page.goto(links[x])
                      await page.waitFor('footer', {visible: true})
                    }
                  })
                })
              })

Right, let’s start with the first problem:
This code has unbalanced block endings.

Lets look at it with the ACTUAL indentation, because my head cant do the whole… left-aligned thing.

const {urls} = require('./common')

describe('Test', async () => {
	describe('url loop',async () => {
		_.forEach(urls, async (url) => {
			it('screenshot', async () => {
				await page.goto(url + '/sitemap')
				let sub_domain = url.replace('https://www.', '')
				let domain = sub_domain.substr(0, sub_domain.indexOf('.co'))
				return Array.from(document.querySelectorAll('#sitemap a')).map(val => val.href);
			});
			for(x=0; x<links.length; x++){
				console.log(x)
				await page.goto(links[x])
				await page.waitFor('footer', {visible: true})
			}
		})
	})
})
})

Why is there an extra }) at the end of the code?
Why have we got a foreach outside of the it declaration?
Why do we not expect anything? What’s the result of this test?

1 Like

Ok thanks for that I will look at it and make some amendments.
The forEach is outside of the ‘it’ because I want to run the test for each specific test.
I want to eventually take screenshots and compare them to previous screenshots taken.
I havent added all the code as this is the code that is causing the issue.

and where are you defining links? I see a urls, but not links.

1 Like

urls are external. links are an array created from the page sitemap. Sorry im an apprentice developer. Really interested in this and always willing to learn.

Where?
I… THINK what you’re trying to do is more properly defined to be:

const {urls} = require('./common')

describe('Test', async () => {
	describe('url loop',async () => {
		_.forEach(urls, async (url) => {
			it('screenshot', async () => {
				await page.goto(url + '/sitemap')
				let sub_domain = url.replace('https://www.', '')
				let domain = sub_domain.substr(0, sub_domain.indexOf('.co'))
				let links = Array.from(document.querySelectorAll('#sitemap a')).map(val => val.href);
				for(x=0; x<links.length; x++){
					console.log(x)
					await page.goto(links[x])
					await page.waitFor('footer', {visible: true})
				}
			})
		})
	})
})

Still not sure I agree with putting a test inside a loop for pure readability concerns, but… okay.

It didnt help much but I added a try catch around the goto and waitfor and found the error

Protocol error (Emulation.setDeviceMetricsOverride): Target closed.

This topic was automatically closed 91 days after the last reply. New replies are no longer allowed.