Best tool for web crawling

General Web Dev
1

Hi,

i want to implement web crawling to get some data from a website.

Which tool is best as well as convenient to start web crawling with?

I am using Ubuntu 17.10 OS running on my laptop.

2

Since you have tagged your post with Python, are you already answering your own question?

3

You can read more about that topic here. Then you will be ready to pick the right tool for you.

4

Bear in mind that thread is from Oct 7 '08 and was closed in 2012, so will not have the most up-to-date information.

5

Usually when someone says “best” in a question they need to define what “best” means for them and they don’t.

Note that that question asks about links specifically. The question here is not specific. The StackOverflow question was also closed so it is limited in responses. The question does say something about being able to work with ASP.Net is best and this question implies that the best solution is not. I could suggest an article I wrote but that is about using C# and Windows. This question does at least indicate that C# and Windows would not be best.

6

This topic was automatically closed 91 days after the last reply. New replies are no longer allowed.