All jobs

Job details

Big Data Engineer (Azure)


About the client

Our client is an advanced analytics consulting firm. They are a trusted partner to renowned analytics companies, enabling them to extract business value from data. The company’s consultants have deep knowledge in data science, machine learning and artificial intelligence.

Responsibilities

  • Architecting Advanced Analytics Solutions: Designing the overall structure and framework for advanced analytics solutions, ensuring scalability, efficiency, and reliability.
  • Designing Analytics Capabilities: Creating designs for batch and streaming analytics systems, ensuring they meet the requirements for processing large volumes of data efficiently.
  • Implementing Analytics Solutions: Developing and deploying machine learning models, natural language generation systems, and other emerging technologies within the realm of advanced analytics.
  • Integration of Technologies: Ensuring seamless integration of various technologies and tools required for advanced analytics, such as cloud services, databases, and analytics platforms.
  • Optimizing Performance: Continuously optimizing the performance of analytics systems to ensure they deliver results in a timely and efficient manner.
  • Ensuring Data Quality and Security: Implementing measures to maintain data quality and security throughout the analytics process, including data ingestion, processing, and analysis.
  • Collaboration and Communication: Collaborating with cross-functional teams to gather requirements, provide technical expertise, and communicate project updates and findings effectively.
  • Staying Updated on Emerging Technologies: Keeping abreast of the latest trends and developments in big data and analytics, and evaluating their potential for incorporation into existing systems.
  • Troubleshooting and Support: Identifying and resolving technical issues that arise during the development and deployment of analytics solutions, and providing support to users as needed.
  • Documentation and Knowledge Sharing: Documenting the design, implementation, and best practices associated with analytics solutions, and sharing knowledge with team members to facilitate learning and collaboration.

Requirements

  • Bachelor’s degree in Computer Science or similar field.
  • 4+ years of experience in traditional and modern Big Data technologies (HDFS, Hadoop, Hive, Pig, Sqoop, Kafka, Apache Spark, hBase, Oozie, No SQL databases).
  • Experience in Java/Python/Scala.
  • Experience extracting/querying/joining large data sets at scale.
  • Experience building data platforms using Azure stack.
  • Experience building data ingestion pipelines using Azure Data Factory to ingest structured and unstructured data.
  • Strong knowledge on Azure Storage schematics such as Gen1 and Gen2.
  • Experience in harmonizing raw data into a consumer-friendly format using Azure Databricks.
  • Knowledge of Azure networking, security, key vaults, etc.
  • Experience in data wrangling, advanced analytic modeling, and AI/ML capabilities is preferred.
  • Experience utilizing Snowflake to build data marts with the data residing in Azure storage is a plus.
  • Strong communication and organizational skills.
  • Upper-Intermediate or higher level of knowledge of English.

We offer

  • Competitive salary and benefits.
  • Flexible working hours.
  • Vacation – 20 working days’ annual leave.
  • Paid sick days.
  • A personal laptop with all the essential software installed.
  • State holidays are granted.
  • Corporate accountant, lawyer, and psychologist services.
  • Bureaucracy-free and transparent management.
  • Costs related to participation in workshops and professional conferences are covered.
  • In-house and online сorporate English classes.
  • Free running lessons with a professional athletics coach.
  • Free online yoga classes.
  • Corporate parties 2 times a year and team-building events

    Apply for this position

    Email address

    Phone number

    First name

    Second name

    Message

    Attach a CV

    Job details

    Big Data Engineer (Azure)

    Remote, Warsaw

    Upper-Intermediate



    Location

    Remote, Warsaw

    Job type

    Full time

    English level

    Upper-Intermediate

    Date posted

    May 16, 2024