Browse
···
Log in / Register

Senior Big Data Engineer

Qode

North Carolina, USA

Favourites
Share

Description

Role Description: Senior Big Data Engineer – Scala/Spark Location: [Charlotte NC/NY/NJ/or Onsite/Hybrid] Type: [Full-time] Experience Level: Senior level only   Role Overview We are looking for a highly skilled Senior Big Data Engineer with strong expertise in Scala and Apache Spark to support our data engineering team. The ideal candidate should be able to read and refactor existing Scala Spark code, develop enhancements, create robust unit tests, and help migrate code into a modern cloud-native environment.   Key Responsibilities: Analyze and refactor existing Scala Spark code to support migration to new environments. Write and enhance Scala Spark code with a focus on performance and maintainability. Develop comprehensive unit tests to ensure code quality and stability. Work with data stored in Amazon S3 (CSV and Parquet formats). Integrate and manipulate data from MongoDB. Independently set up environments to validate and test code changes. Execute both manual and automated testing procedures. Collaborate within an Agile team and contribute to sprint planning, story grooming, and reviews. Maintain clear and proactive communication with team members, project managers, and stakeholders. Technical Skills & Qualifications: Proficiency in Scala and Apache Spark for large-scale data processing. Working knowledge of Python for scripting and data transformation. Hands-on experience with Amazon Web Services (AWS), including: S3 AWS Glue Step Functions Experience working with MongoDB and structured/unstructured data. Familiarity with SQL and BDD frameworks. Experience with Terraform for infrastructure-as-code is a plus. Strong understanding of software development lifecycle (SDLC), from development to production deployment. Experience using tools like Git, Azure DevOps/TFS, and JIRA. Comfortable working in Agile/Scrum environments. Core Technologies: Scala Spark AWS Glue AWS Step Functions Maven Terraform Soft Skills: Strong problem-solving abilities and attention to detail. Ability to work independently and as part of a collaborative Agile team. Excellent written and verbal communication skills.

Source:  workable View Original Post

Location
North Carolina, USA
Show Map

workable

You may also like

iSoftTek Solutions Inc
Senior Software Engineer
Edmond, OK, USA
Senior Software Engineer Job Type: C2C Duration: Long Term Location: Edmond, OK What You’ll Do ●       Develop organizational software for using the appropriate tools and techniques for each platform in span of care. ●       Review and test existing code-base or technical integrations for various projects. ●       Provide support to other Client Engineers by reviewing, testing, and approving code/software development. ●       Collaborate with various teams and team members to determine the technical approach and to organize development plans. ●       Evaluate the complexity of tasks and provide realistic estimates on the level of effort and completion time. ●       Partner with junior and mid-level engineers to offer training and guidance on best practices. ●       Help ensure that other engineers are moving in the same technical direction. ●       Maintain strategic outlook to plan for future development of processes and team. ●       Provide technical support on escalated software issues. ●       Research industry best practices and new technologies or tools that could be implemented at Life.Church. Skills Needed To Succeed ●       Proficient in modern web technologies, such as TypeScript, React, and NextJS. ●       Excellent verbal, written, and interpersonal communication skills to maintain relationships and partnerships. ●       Maintain a systematic and organized approach to completing assignments accurately, thoroughly, and timely. ●       Clearly explain complicated and detailed processes in a concise and simple way. ●       Ability to self-motivate, take initiative, make independent decisions, and problem-solve. ●       Ability to collaborate in a team environment and work independently. ●       Ability to maintain composure during stressful situations. ●       Own technical design for projects and understand, and articulate, the trade-offs in creating high-quality software. ●       High School Diploma or GED. ●       Bachelor’s degree in computer science preferred. ●       4+ years of related work experience.    
Apex Informatics
Sr. Analytics Developer
Atlanta, GA, USA
TheGeorgia Department of Human Services (DHS) is seeking qualified candidates forthe senior analytics Developer positionin Atlanta, Georgia. The desired candidate will design, develop, test, maintain, andsupport complex data extract, transformation, and load (ETL) programs for anEnterprise Data Warehouse. Understanding how complex data should be transformedfrom the source and loaded into the data warehouse is critical to this job. ·Expertiseand solid experience in BI Tools – Power BI and BigQuery ·StrongIDMC(Informatica) technical knowledge in the design, development, andmanagement of complex Informatica mappings, sessions, and workflows ·Strongprogramming skills, relational database skills with expertise in Advanced SQLand PL/SQL, indexing, and query tuning ·Experiencedin Business Intelligence and Data warehousing concepts and methodologies. Contribute to the design and implementation of the solution's metadata, ETL, and reporting components, including testing and, at times, deployment. ·Extensiveexperience in data and root cause analysis and proven problem-solving andanalytical thinking capabilities. ·Analyticalcapabilities to slice and dice data and display data in reports for the bestuser experience. ·Demonstratedability to review and translate business processes into BI reporting andanalysis solutions. ·Abilityto follow the Software Development Lifecycle (SDLC) process and should be ableto work under any project management methodologies used. ·Abilityto follow best practices and standards. ·Abilityto identify and tune BI application performance bottlenecks. ·Abilityto work quickly and accurately under pressure and project time constraints ·Abilityto prioritize workload and work with minimal supervision ·Basicunderstanding of software engineering principles and skills working onUnix/Linux/Windows Operating systems, Version Control, and Office software ·ExposureData Modeling using Star/Snowflake Schema Design, Data Marts, Relational andDimensional Data Modeling, Slowly Changing Dimensions, Fact and Dimensionaltables, Physical and Logical data modeling Produce client deliverables such as detailed design documentation, configuration documentation, unit test plans, and training curriculum Required Qualifications: A bachelor's degree in Computer Science or a related field 6 to 7 years of experience working with Power BI, IDMC,BigQuery and Python Ability to design and develop complex Informatica mappings, sessions, and workflows and identify areas of optimization Experience with Oracle RDBMS 19c Effective communication skills (both oral and written) and the ability to work effectively in a team environment are required. Proven ability and desire to mentor/coach others in a team environment Strong analytical, problem-solving, and presentation skills. Preferred Qualifications: Working knowledge of Informatica Power Exchange/Power Center and IDMC Experience with relational, multidimensional, and OLAP techniques and technology Experience with Visualization tools like MS Power BI, Tableau Experience with BigQuery and Python. Soft Skills: ·        Strong written and oral communication skills in the English Language. ·        Ability to work with Businesses and communicate technical solutions to solve business problems.  
Apex Informatics
Esri ArcGIS Developer
Atlanta, GA, USA
Job Summary: The Georgia Dept. of Transportation (GDOT) is seeking a technical GIS developer for configuring and developing custom Esri ArcGIS Enterprise 10x, 11x (Javascript API, Experience Builder, ArcGIS Portal/Online/Hub, Field Maps) applications. Applications are deployed within SharePoint web sites. Custom GIS solutions will be integrated within .NET based applications, GDOT databases (Oracle/Oracle Exadata, SQL Server), data warehouse (Oracle Exadata), and enterprise reporting (Oracle Business Intelligence Enterprise Edition (OBIEE), Data Visualizer, or Power BI). The candidate will have application programming/development, implementation, and support experience. Under broad supervision, the candidate will design, code, test, modify, and debugs computer software and applications utilizing Esri ArcGIS. The candidate will research and analyze program or systems problems, provides resolutions, and complete system documentation. The candidate will translate business requirements/user stories into development activities in secure, commented, and maintainable code.  The candidate will be working in the Enterprise GIS Team of the Office of IT Applications. The candidate will work with supporting IT team members (database, network, server, etc.) and interact with management, architects, developers, and business units. The candidate will follow internal SLDC, standards/guidelines, and accepted best practices for developing solutions. The Enterprise GIS team is composed of 2 ArcGIS developers, 2 System Administrators, a data manager, and team leader. The team supports over 180 ArcGIS Desktop users and approximately 60,000 annual ArcGIS Server web visits. Applications support GIS portal, feature mapping, project planning, emergency operations, permitting, safety, and traffic records. The team develops enterprise GIS services, applications, and databases. The team coordinates with client, server, network, and database/data warehouse personnel to support and develop the underlying IT solutions. NOTE: While the intent may be a long-term tenure, this position is subject to annual budget restrictions.  The initial contract is through the end of this fiscal year and is anticipated to be renewed July 1, 2025.  Responsibilities: Develops customized web applications, webservices, mobile, and enterprise solutions Develops geoprocessing and GIS web services Develops GIS web editing/mapping applications Conducts performance, unit, system, integration, and quality assurance testing of applications Maintains existing applications/services Upgrades and migrates existing applications/services to current version/systems architecture Provides end user support and creates support documentation Works closely with other developers and architects Estimates work effort and proposes/designs development solutions Work Terms and Conditions: 8 Hour workdays on a fixed schedule (i.e. not a flex-schedule), starting no earlier than 7:00AM Eastern Time, starting no later than 9:00AM Eastern Time, Monday - Friday, 40 hours per week GDOT has NOT specified a number of days onsite vs remote. However, local candidates (i.e. able to come in office) will be preferred over an out of state candidates (i.e. remote worker). GDOT prefers candidates who can demonstrate they have the capability to work a hybrid work schedule which is a combination of remote and in-office work. Candidate capacity (i.e. days they could come into the office) to work a hybrid work schedule will be discussed during interviews. The consultant will follow all applicable State of Georgia and GDOT policies, procedures, standards, guidelines, and accepted best practices. The position end date is set for the end of the GDOT Fiscal Year. GDOT may exercise the option to renew contracts in one-year increments. Several successful consultants have continued working for several years with contract extensions. However, GDOT is under no obligation to extend contracts and may terminate employment at any time. When teleworking or working remotely, consultant must have suitable internet connection and environment to comply with GDOT telework policies. GDOT will reimburse authorized travel expense lodging and meals. GDOT will reimburse authorized travel use of a personal vehicle. Any travel expenses to start work or relocate to Georgia or will not be reimbursed. All travel expense reimbursement shall be in accordance with State Accounting Office Travel Policy: https://sao.georgia.gov/travel/state-travel-policy The Department has 13 holidays, which consultants are not allowed to work nor make up any time for those holiday weeks without management's approval. Consultants are not to take extended leave during their first year of service (e.g. greater than 1week). Consultant must have cell phone which can be used for business calls. GDOT does not provide free parking. All routine commuting expenses are the responsibility of the consultant.
Cookie
Cookie Settings
Our Apps
Download
Download on the
APP Store
Download
Get it on
Google Play
© 2025 Servanan International Pte. Ltd.