/spider

A spider project, fetch short-term housing information from a website and save the data into Mongo database.

Primary LanguagePythonMIT LicenseMIT

Spider

license

Writing a website spider is always a good way for those who have just learned a new programming language and want to train themselves by writing some small projects. And a spider project is simple and small enough and also can use most of the features in the new language as you want. So I have created the project Sprider here, just for fun 😄.

Table of Contents

Python

Short-term housing information is a kind of information whose who are willing to find a place to live in for not too short, but not too long time. The duration is embarrassed if you just want to save some money, for you cannot stay in hotel for a long time, or rent a apartment.

So for the python version web spider, the spider will fetch information automatically from xiaozhu, and store the data into Mongo database. And then, show the data in a web page.

Go

For the go version, we will acquire some beautiful girl's pictures form a website, and then download and save them. In order to speed up, we will use goroutine and channel.

License

MIT.