summaryrefslogtreecommitdiff
path: root/crawler.py
Commit message (Collapse)AuthorAgeFilesLines
* rewrite in NodeJSschneefux2017-03-281-90/+0
|
* use 2.0 joblibschneefux2017-03-271-40/+32
|
* do not crash on errorrelease/1.2.0schneefux2017-03-181-1/+1
|
* hotfix: do not retry foreverVainsocial root2017-03-121-0/+6
|
* sleep less after ratelimitschneefux2017-03-101-1/+1
|
* fail on API server errorschneefux2017-03-101-1/+6
|
* log: be more verboseschneefux2017-03-081-2/+0
|
* log API errorsschneefux2017-03-071-3/+2
|
* fix 'unknown encoding' bugschneefux2017-03-071-0/+1
|
* tolerate API json errorschneefux2017-03-051-1/+3
|
* run jobs in batches (#57)schneefux2017-03-051-0/+1
|
* don't fail with 404 if a player was foundschneefux2017-02-271-0/+5
|
* set api pagelimit to maximumschneefux2017-02-271-1/+1
|
* Merge branch 'errors'schneefux2017-02-271-2/+8
|\
| * catch more API bugsschneefux2017-02-261-1/+4
| |
| * report job fails in queueschneefux2017-02-261-1/+4
| |
* | Merge pull request #39 from vainglorygame/redesignKapil Viren Ahuja2017-02-261-64/+24
|\| | | | | rewrite to use shared queue
| * rewrite to use shared queue, fixes #36schneefux2017-02-251-64/+24
| |
* | update limits: cannot get more than 5 random matchesschneefux2017-02-251-2/+2
|/
* use gzip for API requestsschneefux2017-02-251-1/+1
|
* respect rate limitsschneefux2017-02-251-5/+16
|
* add VAINSOCIAL_APIKEY env variableschneefux2017-02-251-3/+2
|
* catch ssl RuntimeError and retry (#24)schneefux2017-02-161-1/+1
|
* ignore server errorsschneefux2017-02-151-4/+9
|
* rewrite for better concurrency supportschneefux2017-02-121-2/+0
|
* use logging moduleschneefux2017-02-081-1/+2
|
* crawl regions peux a peuxschneefux2017-02-061-4/+16
|
* api: get new matches more reliably, remove now unused `db.meta`schneefux2017-01-281-2/+1
|
* api: demo /matches endpointschneefux2017-01-271-0/+7
|
* api: fix getting new matchesschneefux2017-01-231-0/+2
|
* api: send requests synchronouslyschneefux2017-01-231-23/+15
|
* api: rewrite to use asyncschneefux2017-01-221-18/+35
|
* api: implement crawler and database functionalityschneefux2017-01-221-0/+72