Senior Databricks Solution Architect

3 giorni fa

Gallicano nel Lazio RM, Provincia di Roma; Lazio, Italia JobLeads Tempo pieno

We are seeking a Senior Databricks Architect to design, implement, and optimize enterprise-scale data platforms using theDatabricks Lakehouse Platform. The ideal candidate will have deep expertise in Databricks architecture, Apache Spark, Delta Lake, data engineering, cloud platforms, and enterprise data solutions.

Pronto/a a candidarsi? Prima di farlo, si assicuri di leggere tutti i dettagli relativi a questo lavoro nella descrizione sottostante.
Key Responsibilities

  • Design and architectDatabricks Lakehouse solutionsfor large-scaleenterprise data platforms.
  • Lead end-to-end Databricks implementation, including architecture, development, deployment, and optimization.
  • Define Databricks workspace architecture, clusterstrategies, security models, and operational standards.
  • Design and developscalable data pipelinesusingDatabricks, Apache Spark, PySpark, SQL, andDelta Lake.
  • Implement data ingestionframeworks supporting batchand real-timedata processing.
  • Design Delta Lake architectures including:
  • Delta tables
  • Medallion architecture (Bronze/Silver/Gold layers)
  • Data quality frameworks
  • Schema evolution andoptimization
  • Perform Databricks performancetuning:
  • Cluster optimization
  • Spark job optimization
  • Query tuning
  • Cost optimization
  • Implement Databricks governanceand security using:
  • Unity Catalog
  • Access controls
  • Data lineage
  • Audit capabilities
  • Lead migration projectsfrom legacy dataplatforms to DatabricksLakehouse.
  • Define best practicesfor Databricks CI/CD, DevOps, andenvironment management.
  • Collaborate with dataengineers, data scientists, analysts, and enterprisearchitects.
  • Provide technical guidance, architecture documentation, and design reviews.

Skills & Experience

  • Strong experience in dataengineering, big data, or solution architecture.
  • Strong hands-onexperience withDatabricks Platform.
  • Expert knowledge of:
  • Apache Spark
  • PySpark / Scala
  • Delta Lake
  • SQL
  • Data Lakehouse architecture
  • ETL/ELT development
  • Experience designing anddelivering enterprise Databrickssolutions.
  • Strong experience withcloud platforms:
  • Azure Databricks (preferred)
  • AWS Databricks
  • Google Cloud Databricks
  • Experience with:
  • Databricks Workflows
  • Jobs orchestration
  • Notebooks
  • Repos
  • Cluster management
  • Unity Catalog

Preferred Skills

  • Databricks Certified DataEngineer Professional /Databricks Certified Architect.
  • Experience with:
  • MLflow
  • Databricks Machine Learning
  • Feature Store
  • Model deployment
  • MLOps
  • Knowledge of streamingtechnologies:
  • Structured Streaming
  • Kafka
  • Experience with datagovernance, compliance,and enterprise securityframeworks.

Key Competencies

  • Strong Databricks architectureand implementation expertise.
  • Ability to designscalableand cost-efficientLakehouse solutions.
  • Strong troubleshooting andperformance optimizationskills. xrdztoy
  • Ability to leadtechnical discussions withenterprise stakeholders.