Big Data Engineer

In the Market Regulation Surveillance Patterns technology team, an expert in this role is vested with responsibilities arising from FINRA’s mission to protect the integrity of US Securities Capital markets. FINRA has a portfolio of ‘surveillance patterns’ that look for manipulative and non-compliant behavior in the database of all transactions that occur in the stock market. The database itself consists of tens of billions of records per day. Engineers work with these large volumes of data using state-of-the-art and industry standard technologies, all of which are wholly operated in a cloud computing environment.

In order to operate with ever improving effectiveness, the team operates in a rich culture of performance and innovation. Collaboration within the team and with business stakeholders is frictionless, with significant independence offered in order to experiment with better ways of conducting technology for business value. Technological and career growth opportunities are a natural and every day part of the working environment.

Please read further to get a better feel for the role and the skills that lead to success in this team.

Job Functions:

• Analyze system requirements and design responsive algorithms and solutions

• Use big data and cloud technologies to produce production quality code

• Engage in performance tuning and scalability engineering

• Work with team, peers and management to identify objectives and set priorities

• Perform related SDLC engineering activities like sprint planning and estimation

• Work effectively in small agile teams

• Provide creative solutions to problems

• Identify opportunities for improvement and execute

Essential skills:

• Experience with cloud based Big Data technologies

• Proficiency in Hive / Spark SQL

• Experience with Spark

• Experience with one or more programming languages like Scala, Python, and/or Java

• Ability to push the frontier of technology and independently pursue better alternatives

Requirements

Essential skills:

• Experience with cloud based Big Data technologies

• Proficiency in Hive / Spark SQL

• Experience with Spark

• Experience with one or more programming languages like Scala, Python, and/or Java

• Ability to push the frontier of technology and independently pursue better alternatives

Essential skills: Experience with cloud based Big Data technologies Proficiency in Hive / Spark SQL Experience with Spark Experience with one or more programming languages like Scala, Python, and/or Java Ability to push the frontier of technology and independently pursue better alternatives