Credit card transactions web scraper
The company behind my credit card provides a web interface which includes a log of the transactions that have been made. The objective of this project is to automatically retrieve this log everyday and store its data in a database so as to display it on a custom dashboard.
The main challenge of this project is the access to the log, which requires user authentication. Luckily, the security of the web interface is quite loose and consists only of a username-password combo. Thanks to this, PhantomJS can be used to log into the web interface and retrive the log data without needing human interaction. Note that the development of PhantomJS is currently suspended until further notice.
Here, PhantomJS is used as a Node.js module so that the web scraping code can be easily integrated with that of the database access. The scraping results are stored in a MongoDB database, which can be access by a finance management app I developed separately.
Source code available on GitHub