Senior Data Engineer
před 5 dny

Job Description : What you will do

What you will do

  • Design & implement data ingestion and processing of various data sources using cloud (MS Azure, AWS, Google) big data’ technologies like Spark, DataBricks, Glue, Airflow, Kafka, DataFactory, NoSql DBs, SageMaker, ML Studio
  • Create and maintain data tools for data scientist / analyst teams that assist them in building and optimizing data algorithms like AI / Machine Learning models, productionize the models
  • Assemble large, complex data sets that meet functional / non-functional business requirements for data lakehouse
  • Develop data pipelines to provide actionable insights into marketing automation, customer acquisition, and other key businesses
  • Deploy DevOps automation of continuous development / test / deployment processes
  • Work with stakeholders to assist with data-related technical issues and support their data infrastructure needs, like optimizing existing data delivery, re-designing infrastructure for greater scalability, etc.
  • Support pre-sales by proposing technical solution and accurate effort estimate
  • Required Skills

  • Experience in building and productionizing big data architectures, pipelines and data sets.
  • Understanding (big) data concepts and patterns (data lake, lambda architecture, streaming processing, DWH, BI & reporting)
  • 2+ years of experience in a Data Engineer role, who has attained experience using the following software / tools :
  • Experience with big data tools : Hadoop, Spark, Kafka, etc.Experience with object-oriented / object function scripting languages : Python, Scala, Java, Scala, R, C++ etc.
  • Experience with MS Azure (DataBrics, Data Factory, Data Lake, Cosmos DB, Event Hub, PowerBI) or AWS (Glue, EC2, EMR, RDS, Redshift, Sagemaker) cloud servicesImplementing large-scale data / events oriented pipelines / workflows using ETL toolsExtensive working experience with relational (MS SQL, Oracle, postgress, Snowflake.

  • and NoSQL databases (Cassandra, MongoDB, Redshift, Elasticsearch, Redis, )
  • Strong analytic skills related to working with (un)structured datasets.
  • Build processes supporting data transformation, data structures, metadata, dependency and workload management.
  • Experience in setting up and using CI / CD automation tools
  • Strong project management and organizational skills.
  • Preferred Skills

  • Deep hands-on development experience in MS Azure or AWS environments
  • Past experience in delivery of business intelligence projects, using tools like PowerBI, Tableau, Qlick Sense, Keboola
  • Working knowledge of message queuing, stream processing, and highly scalable real-time data processing using technologies like Storm, Spark-Streaming, etc.
  • Experience with data pipeline / workflow management tools like Airflow, NiFi, StreamSets, Glue, Azure Data Factory etc.
  • Qualifications :

    Nahlásit tuto nabídku

    Thank you for reporting this job!

    Your feedback will help us improve the quality of our services.

    Můj e-mail
    Kliknutím na "Pokračovat", souhlasíte s tím, že neuvoo sbírá a zpracovává vaše osobní údaje, které jste poskytli v tomto formuláři, aby vytvořili neuvoo účet a přihlásili vás k odběru emailových upozornění v souladu s naší Ochranou Osobních Údajů . Váš souhlas můžete vzít kdekoliv zpět, následováním těchto kroků .