Skip to content

vincom2/python-NLB-scraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

12 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Singapore NLB catalogue scraper

This uses requests/Beautiful Soup to scrape NLB's catalogue, and web.py to serve it up. At the moment, it only does the bog-standard search, which means (I think) search by keyword, no limiting by branch or whatnot.
serve-local.py is for non-WSGI use. serve.py is for WSGI use (well, I think. Bear in mind I don't actually know what I'm talking about; I pieced this shit together after frustrating minutes googling for a lot of different crap). Yes, there probably is a better way of sticking them in the same file, or at least, if they're going to be different files, avoid the complete copypasta, but I'm lazy and don't know how ifdefs (do they exist for this stuff?) or Python modules work. So thar :P


###Advantages: Uh... well, for waiting a little longer, you get a plaintext list of all results for your keyword(s). I don't believe this is possible on the NLB catalogue site proper, which is kind of annoying because their keyword search is not very good and so it can be useful to be able to Ctrl+F the results. I don't think there are any other advantages, lol, unless you particularly enjoy looking at absolutely bare pages over fancy ones. I have other ideas for this, but we'll see if I ever get around to implementing any of them...


###Usage:

Non-WSGI:

Put serve-local.py and templates/ (and yes, the .html files inside templates/, obviously) into the same directory. Then

python serve-local.py

Then visit 127.0.0.1:8080 in your browser.

WSGI/App Engine (I don't actually know):

Just... upload it? Everything's here already, lol. Oh, but you have to take note of silly things like this. Meh.

Oh, also, Python 2.7, preferably.


Try it for yourself! [http://nlbcatalogue.appspot.com/]

About

python code to scrape Singapore NLB catalogue

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages