Senior Databricks Solution Architect

7 giorni fa

Gallicano nel Lazio, Lazio, Italia JobLeads Tempo pieno

We are seeking a Senior Databricks Architect to design, implement, and optimize enterprise-scale data platforms using theDatabricks Lakehouse Platform. The ideal candidate will have deep expertise in Databricks architecture, Apache Spark, Delta Lake, data engineering, cloud platforms, and enterprise data solutions.



Pronto/a a candidarsi? Prima di farlo, si assicuri di leggere tutti i dettagli relativi a questo lavoro nella descrizione sottostante.
Key Responsibilities

  • Design and architectDatabricks Lakehouse solutionsfor large-scaleenterprise data platforms.
  • Lead end-to-end Databricks implementation, including architecture, development, deployment, and optimization.
  • Define Databricks workspace architecture, clusterstrategies, security models, and operational standards.
  • Design and developscalable data pipelinesusingDatabricks, Apache Spark, PySpark, SQL, andDelta Lake.
  • Implement data ingestionframeworks supporting batchand real-timedata processing.
  • Design Delta Lake architectures including:

    • Delta tables
    • Medallion architecture (Bronze/Silver/Gold layers)
    • Data quality frameworks
    • Schema evolution andoptimization

  • Perform Databricks performancetuning:

    • Cluster optimization
    • Spark job optimization
    • Query tuning
    • Cost optimization

  • Implement Databricks governanceand security using:

    • Unity Catalog
    • Access controls
    • Data lineage
    • Audit capabilities

  • Lead migration projectsfrom legacy dataplatforms to DatabricksLakehouse.
  • Define best practicesfor Databricks CI/CD, DevOps, andenvironment management.
  • Collaborate with dataengineers, data scientists, analysts, and enterprisearchitects.
  • Provide technical guidance, architecture documentation, and design reviews.

Skills & Experience

  • Strong experience in dataengineering, big data, or solution architecture.
  • Strong hands-onexperience withDatabricks Platform.
  • Expert knowledge of:

    • Apache Spark
    • PySpark / Scala
    • Delta Lake
    • SQL
    • Data Lakehouse architecture
    • ETL/ELT development

  • Experience designing anddelivering enterprise Databrickssolutions.
  • Strong experience withcloud platforms:

    • Azure Databricks (preferred)
    • AWS Databricks
    • Google Cloud Databricks

  • Experience with:

    • Databricks Workflows
    • Jobs orchestration
    • Notebooks
    • Repos
    • Cluster management
    • Unity Catalog


Preferred Skills

  • Databricks Certified DataEngineer Professional /Databricks Certified Architect.
  • Experience with:

    • MLflow
    • Databricks Machine Learning
    • Feature Store
    • Model deployment
    • MLOps

  • Knowledge of streamingtechnologies:

    • Structured Streaming
    • Kafka

  • Experience with datagovernance, compliance,and enterprise securityframeworks.

Key Competencies

  • Strong Databricks architectureand implementation expertise.
  • Ability to designscalableand cost-efficientLakehouse solutions.
  • Strong troubleshooting andperformance optimizationskills. xbvzest
  • Ability to leadtechnical discussions withenterprise stakeholders.