ANDREW I DONT CARE ABOUT YOUR STORIES! JUST GIVE ME THE LINK! >> http://andrewmohawk.com/pasteLert/
So here is my latest project, extending from the previous pasteScraper to do something a little different with the pastebins. Essentially i recreated google alerts but with a bit more searchiness (yes, i make up words now too).
How it Works
- I enumerate all new pastes from http://www.pastebin.com/archive/ every minute and add them to a ‘download’ queue.
- New pastes are then downloaded to a local database
- Alerts are periodically cron’d
- Search functionality is via a fulltext search of pastes
What does it give me?
- The ability to search for *anything* on pastebin.com
- Semi-realtime searches
- Email alerts when your term is hit!
- RSS feeds for searches
- The ability to search with AND keywords in pastebins
How it is all going to fall apart
I dont really see this as a long term project, merely something that shows a PoC for how much stuff is leaking out via PasteBin.com and how cool it really is. Some issues i see that may happen with this:
- People will switch to more secure pastebins that don’t allow indexing, don’t have archive pages and arent indexed by search engines
- My small linode will fall to pieces because the fulltext like queries are painfull
- Pastebin.com will not be impressed with me doing this and start blocking it
http://andrewmohawk.com/pasteLert/, feel free to play/comment/etc :)
p.s. Thanks to Chris Hadnagy and Roelof Temmingh :D