Python programmer állások XIV. kerületben (adatbázisszakértő)

• Design and build scalable, reusable frameworks and services for data ingestion, transformation, and processing
• Develop and maintain the platform’s core capabilities such as orchestration, data quality validation, and schema management tools

• Maintaining and evolving data structures
• Building new ETL flows and maintaining or improving existing ones
• Designing new data structures and data modelling concepts
• Ensuring data quality by verifying data consistency and accuracy
• Creating data validation methods

• Maintaining and evolving data structures
• Building new ETL flows and maintaining or improving existing ones
• Designing new data structures and data modelling concepts
• Ensuring data quality by verifying data consistency and accuracy
• Creating data validation methods

• Build, operate, and improve production data pipelines using Python, Airflow, Kubernetes, and related technologies
• Design scalable data pipelines and data models for growing data volumes and processing needs

• Establish data feeds with custodian banks, data vendors, and other service providers
• Support senior leadership, clients, and the broader Data Team in executing analyses and providing data-driven answers to questions
• Fully automate and centralize existing data integration processes

Working on the design, development, and production of advanced digital solutions, ensuring seamless integration with existing infrastructure and adherence to enterprise architecture standards and robust delivery lifecycle processes. This includes leveraging expertise in various digital technologies...

• Technikai menedzsment és fejlesztés a csapat felhőalapú adatplatformján és adatmodellein Azure Databricks környezetben
• A Data-Driven Project Modelling (DPM) motor fejlesztése és karbantartása DevOps legjobb gyakorlatok követésével

• Development, testing, and maintenance of data pipelines to extract, transform, and load (ETL) data from various sources
• Writing high-quality, maintainable Python code
• Participation in code reviews
• Participation in technical discussions for tool selection and architectural decisions

• Data pipeline development
• Data integration
• Data warehousing
• Data modelling
• Data quality and governance
• Performance optimization
• Collaboration with data scientists and analysts
• Technology selection
• Security and compliance
• Monitoring and troubleshooting

• Design and implement pipelines to ingest, clean, and transform data into Data Lakes and Data Warehouses
• Active monitoring and optimization of pipelines for performance and cost
• Configuration of pipeline deployments and management of environment settings using IaC tools (Terraform) and CI/CD

• Lead the design and implementation of scalable data pipelines and Lakehouse architectures
• Act as the design authority
• Solve the hardest technical challenges including optimizing high-load streaming pipelines, debugging complex Spark jobs, and designing generic frameworks