Hey everyone. I have an issue with the Python library Scrapy, to scrape glassdoor.com
I get the users location and job, and that works great. However, the issue I have is scraping the “Job Count” number. As an example: “https://www.glassdoor.com/Job/jobs.htm?sc.keyword=web%20developer&locT=C&locId=1147070&locKeyword=Coachella,%20CA&srs=RECENT_SEARCHES”
The job count number here is listed as “10 jobs”, however every time I try and scrap this, it will give me a different number each time. It changes between 6 to 10. I tried adding a delay to the scrap, thinking it might be because the value is generated from a JS script. However the delay did not solve it.
I just get the number from the response.css =
Does anyone have any idea, or can point me in the right direction? I’m running out of ideas to try :x