A financial data dashboard

Developed as part of my 3-month Data Engineer course at Datascientest, in parallel to the pedagogical cursus.

It basically consists of :

• Retrieving stock market data at regular intervals,
• Storing it in a local database,
• Making it available through an HTTP API
• Displaying the data in a dashboard
• Running that in production in a microservices-way

You can also see :

• A live version of the data dashboard, running on a very slow (but free) Oracle Cloud instance
• A live version of the API documentation (use "guest"/"guest" as username/password)
• The Github repo with source code, documentation, tests, ...
• A PDF report
• A presentation in slides

Technical stack

Dashboard

Dash

API server

FastAPI

Stock market data storage

MongoDB

Deployment

Docker

Data validation

Pydantic

Data processing

Pandas

Unit/integration/e2e testing

Pytest

Configuration management

Dynaconf

A Firefox extension for job search

I have developed this extension to assist me in my job search and as a demonstration of web-scraping on a non-friendly site (LinkedIn).

It is capable of :

• Automatically extracting data from LinkedIn job offer pages (e.g. job position, company name, link, company domain, location, ...),
• Saving it in a plain text database,
• Displaying a colored overlay on offers depending on their status (seen/applied to/dismissed/rejected/...)
• Displaying the offers in a sidebar

It was also a good excuse to dive into PureScript, a Haskell-like pure functional language that compiles to Javascript, which just like its elder has tons of fascinating programming concepts.

Resources :

• The Github repo

Technical stack

Frontend

Purescript

Architecture

Web Extensions

Backend

Python

Database

Recutils

A monitor of French political shows

A project that aims at automatically detecting who was invited on some French political radio shows, with inspiration from Politiquemedia.

This application :

• has run 24/7 since 2021,
• regularly polls an inventory of various sources with vastly different layouts (xml podcast feed, html pages, ...)
• extract relevant information (using notably Natural Language Processing / Entity Recognition)
• stores it in a structured SQLite database
• presents it on a website, with pages that present data in various ways : by show, by day, by person invited, ...

Please note that although the application is still running and doing its job for some of the sources, the ever-evolving nature of the data processed (changes in xml/page layout, mostly) requires regular updates which I have not been able to provide.

With that in mind, you can see :

• A live version
• The Github repo

Technical stack

Backend

Elixir

HTTP Server

Elixir Phoenix

Web Frontend

Elixir Phoenix LiveView

Data storage

SQLite

Natural Language Processing

Spacy

A MVP for a social network

I developed this MVP all by myself as a demonstration platform with the intention of building a business around it.

It had the following features, with a backend server and a responsive Web client :

• User registration and administration
• Management of social contacts (friendship, groups, ...)
• Live chat and messaging
• Management of a private library of links that could be annotated or shared with contacts
• Client-side notifications

The live version has been put down and the code remained closed source.

Technical stack

Backend

Elixir

HTTP Server

Elixir Phoenix

Web Frontend

Elixir Phoenix LiveView

Data storage

PostgreSQL

Deployment

Ansible

Client-side notifications

Native Javascript