Node JS Nested loop. Iteration not completing

JavaScript
1

I have an array of website urls. I am looping each with another loop for links form its sitemap.
The code I have works but finishes the inner sitemap loop way too early (after 5 iterations) and goes to the next url.

Modules I am usng are puppeteer, mocha lodash and fs extra.
I attempted lodash in the second loop but received a memory leak so I went for a standard loop.

const {urls} = require('./common')

 describe('Test', async () => {
      describe('url loop',async () => {
         _.forEach(urls, async (url) => {
            it('screenshot', async () => {
                await page.goto(url + '/sitemap')
                let sub_domain = url.replace('https://www.', '')
                let domain = sub_domain.substr(0, sub_domain.indexOf('.co'))
                    return Array.from(document.querySelectorAll('#sitemap a')).map(val => val.href);
                  });

                  let links = await page.evaluate(() => {
                      return Array.from(document.querySelectorAll('#sitemap a')).map(val => val.href);
                      });


                    for(x=0; x<links.length; x++){
                      console.log(x)
                      await page.goto(links[x])
                      await page.waitFor('footer', {visible: true})
                    }
                  })
                })
              })
2

Right, let’s start with the first problem:
This code has unbalanced block endings.

Lets look at it with the ACTUAL indentation, because my head cant do the whole… left-aligned thing.

const {urls} = require('./common')

describe('Test', async () => {
	describe('url loop',async () => {
		_.forEach(urls, async (url) => {
			it('screenshot', async () => {
				await page.goto(url + '/sitemap')
				let sub_domain = url.replace('https://www.', '')
				let domain = sub_domain.substr(0, sub_domain.indexOf('.co'))
				return Array.from(document.querySelectorAll('#sitemap a')).map(val => val.href);
			});
			for(x=0; x<links.length; x++){
				console.log(x)
				await page.goto(links[x])
				await page.waitFor('footer', {visible: true})
			}
		})
	})
})
})

Why is there an extra }) at the end of the code?
Why have we got a foreach outside of the it declaration?
Why do we not expect anything? What’s the result of this test?

1 Like
3

Ok thanks for that I will look at it and make some amendments.
The forEach is outside of the ‘it’ because I want to run the test for each specific test.
I want to eventually take screenshots and compare them to previous screenshots taken.
I havent added all the code as this is the code that is causing the issue.

4

and where are you defining links? I see a urls, but not links.

1 Like
5

urls are external. links are an array created from the page sitemap. Sorry im an apprentice developer. Really interested in this and always willing to learn.

6

Where?
I… THINK what you’re trying to do is more properly defined to be:

const {urls} = require('./common')

describe('Test', async () => {
	describe('url loop',async () => {
		_.forEach(urls, async (url) => {
			it('screenshot', async () => {
				await page.goto(url + '/sitemap')
				let sub_domain = url.replace('https://www.', '')
				let domain = sub_domain.substr(0, sub_domain.indexOf('.co'))
				let links = Array.from(document.querySelectorAll('#sitemap a')).map(val => val.href);
				for(x=0; x<links.length; x++){
					console.log(x)
					await page.goto(links[x])
					await page.waitFor('footer', {visible: true})
				}
			})
		})
	})
})

Still not sure I agree with putting a test inside a loop for pure readability concerns, but… okay.

7

It didnt help much but I added a try catch around the goto and waitfor and found the error

Protocol error (Emulation.setDeviceMetricsOverride): Target closed.
8

This topic was automatically closed 91 days after the last reply. New replies are no longer allowed.