AWS Databricks

**Please strictly adhere to the following resume naming convention:
ALL CAPS NO SPACES B/T UNDERSCORES

PTNUSGBAMSREQIDCandidateBeelineID
i.e. PTNUS9999999SKIPJOHNSON0413

Bill Rate: $70-75/hr

MSP Owner: Brandon Wyatt
Location: Quincy MA - onsite
Duration: 6 months
GBaMS ReqID:

AWS Databricks

Role Descriptions: What you will be responsible forAs a Data Engineer you willUse your understanding of large scale data processing and analytics to wrangle our unique cybersecurity data and create automation analyses and tools that point to the most significant business governance and risk management in the design and buildout of petabyte scale systems for high availability high throughput data consistency security and end user privacy defining our next generation of data analytics toolingBuild data modeling automation and ELT workflows to produce Raw Rationalized co-Related and Reporting data flows for graph timeseries structured and semi-structured cybersecurity dataEducation QualificationsMinimum QualificationsB.S. or M.S. in Computer Science or equivalent work experience5 years of experience building large scale distributed systems and data analytics processes on cloud native in-memory and fit-for-purpose hybrid infrastructure. Experience with cybersecurity data and globally distributed log event processing systems with data mesh and data federation as the architectural core is highly in big data technologies like PrestoTrino Spark Flink Airflow Prefect RedPanda Kafka Iceberg Delta Lake Snowflake Databricks MemGraph Neo4J as well as modern security tooling like Splunk Panther Datadog Elastic Arcsight designing and building data warehouse data lake or lake house using batch streaming lambda and data mesh solutions and with improving efficiency scalability and stability of system working with data warehouses or Databases like Snowflake Redshift Postgres Cassandra etcExperience writing and optimizing complex SQL and ETL development and designing and building data warehouse data lake or lake house solutions. Experience building Data APIs and integrations using tools like GraphQL Apache Arrow gRPC ProtoBuf designing large scale stream processing systems with Flink Kafka NiFI and similar with distributed systems and distributed data storage and large-scale data warehousing solutions like BigQuery Athena Snowflake Redshift Presto etc. Experience working with large datasets and best in class data processing technologies for both stream and batch processing graph and time series data notebooks and analytic visualization communication and collaboration skills particularly across teams or with functions like data scientists or business Experience5 years of experience with Python Java or similar languages with cloud infrastructure (e.g. AWS GCP Azure) and deep experience working with big data processing infrastructures and ELT orchestrationExperience developing distributed batch and real-time feature stores and developing coordinated batch streaming and online model execution workflows building and optimizing large scale data proc
Essential Skills: What you will be responsible forAs a Data Engineer you willUse your understanding of large scale data processing and analytics to wrangle our unique cybersecurity data and create automation analyses and tools that point to the most significant business governance and risk management in the design and buildout of petabyte scale systems for high availability high throughput data consistency security and end user privacy defining our next generation of data analytics toolingBuild data modeling automation and ELT workflows to produce Raw Rationalized co-Related and Reporting data flows for graph timeseries structured and semi-structured cybersecurity dataEducation QualificationsMinimum QualificationsB.S. or M.S. in Computer Science or equivalent work experience5 years of experience building large scale distributed systems and data analytics processes on cloud native in-memory and fit-for-purpose hybrid infrastructure. Experience with cybersecurity