Yassine Jouini

As Architect:

• Defined integration and data processing patterns in batch and streaming modes.
• Conducted a feasibility study on switching to a dataflow solution (Nifi) for existing processing.
• Implemented Spark best practices.
• Conducted impact studies for the deployment of new solutions.
• Served as the primary point of contact for Cloudera support.
• Analyzed the application and system impact of migrating from Cloudera 5 to CDP 7.
• Set up a Pre-Prod cluster from scratch.
• Studied the migration of Spark Streaming jobs to Nifi or Flink.
• Conducted a study on migrating projects from Cloudera to GCP.

As Data Tech Lead:

• Upskilled two data engineers on Big Data technologies.
• Optimized existing codebases.
• Performed code audits.

As Data Engineer:

• Migrated a Cloudera-based project (HDFS ingestion and Impala webservice) to an AWS Cloud architecture (Ingestion to S3, processing with Glue and Lambda, and a webservice using Athena).
• Developed a webservice using Spring Boot.
• Developed Spark batch jobs orchestrated by Oozie.
• Developed Spark streaming jobs.
• Orchestrated jobs using Oozie.
• Designed a dynamic dashboard.
• Migrated a project using Flume to a Nifi solution (developed a specific processor in Nifi).

DevOps and Administration:

• Provided administration and support for CDH5 and CDP clusters.
• Automated installations via Ansible.
• Migrated the cluster from CDH 5.12 to CDH 5.16, and subsequently to CDP 7.1.7.
• Technical Environment: AWS (S3, Athena, Lambda), Spring, Scala, Cloudera 5.12/5.16/7.1.7, Ansible, Python, Linux, Kerberos, HDFS, Hive, Spark, Oozie, Kafka, Tableau, Zeppelin.

As Architect:

• Defined new architecture for the AXA Germany entity; designed a specific Kafka Confluent solution tailored to their needs.
• Managed security aspects (Kerberos and LDAP with specific Cloudera development).
• Implemented Spark and Impala best practices.
• Conducted architecture studies for new applications.
• Acted as the point of contact for Cloudera support.

DevOps and Administration:

• Administered and supported Cloudera and Elasticsearch clusters.
• Aligned different solutions across various environments.
• Automated R package deployment and administration tasks using Ansible and Python.
• Provided support for Hadoop, HDFS, Hive, Spark, Oozie, Flume.
• Security: Provided support on LDAP and Kerberos.
• Performed code audits.
• Technical Environment: Elasticsearch, Kafka, Cloudera CDH 5.8, Ansible, Python, Cloudera Manager, Linux, Kerberos, HDFS, Hive, Spark, Oozie.

POC Phase:

• Installed Cloudera and Elasticsearch clusters.
• Developed a Beats/Logstash job to ingest data from various sources into Kafka.
• Developed Spark Streaming jobs (Java, Python) to import data from Kafka to Elasticsearch.
• Implemented a consolidation job (OrientDB).
• Implemented the Search page and 360 View (ReactJS and NodeJS).

Project Phase:

• Implemented the project using the Exalead solution (selected for data processing).
• Visualized data using Exalead.

Cross-Channel POC (Tunisia Entity):

• Imported customer actions via Google Tag Manager from digital channels (web portals, e-shops) and social networks (Twitter, Facebook).
• Integrated data into Kafka.
• Processed data from Kafka using Spark Streaming for customer reconciliation into Hive.
• Reporting using Zeppelin.
• Technical Environment: Elasticsearch, Kafka, CDH 5.7.0, Hive, HDFS, Spark, ReactJS, Zeppelin, Scala, Python, Java, Javascript, MongoDB, NodeJS, ELK Suite, OrientDB.

• Analyzed server logs from websites to detect customer behaviors and adapt commercial offers (POC context).
• Imported and parsed server data via Logstash to Kafka.
• Developed a Spark Streaming job for data cleaning, followed by integration into Elasticsearch for analysis and Hadoop for storage.
• Created KPI presentations and reporting using Zeppelin via Hive.
• Technical Environment: Spark, Hive, Zeppelin, Logstash, Kafka, Java.

• Imported social media data (Twitter, Facebook, YouTube) into Kafka.
• Developed using Spark Java (Batch).
• Used Spark Streaming for real-time data extraction and enrichment from Kafka to Elasticsearch.
• utilized Machine Learning with Spark MLlib (Scala) for real-time text sentiment analysis.
• Used Elasticsearch and Kibana for visualization.
• Coordinated batch and streaming workflows with Oozie.
• Technical Environment: Elasticsearch, Spark, Hive, Storm, Oozie, Logstash, Kafka, Scala, Python, Java.