Close
Open House on the 24th, - 12 mid day to 5 pm.

Senior Engineer – Hadoop, Pyspark

Category: Product ID: 1151

Description

About the job

You Lead the Way. We’ve Got Your Back.

At American Express, we know that with the right backing, people and businesses have the power to progress in incredible ways. Whether we’re supporting our customers’ financial confidence to move ahead, taking commerce to new heights, or encouraging people to explore the world, our colleagues are constantly redefining what’s possible — and we’re proud to back each other every step of the way. When you join #TeamAmex, you become part of a diverse community of over 60,000 colleagues, all with a common goal to deliver an exceptional customer experience every day. We back our colleagues with the support they need to thrive, professionally and personally. That’s why we have Amex Flex, our enterprise working model that provides greater flexibility to colleagues while ensuring we preserve the important aspects of our unique in-person culture. Depending on role and business needs, colleagues will either work onsite, in a hybrid model (combination of in-office and virtual days) or fully virtually. You won’t just shape the world of software.

You’ll shape the world of life, work and play. Our Software Engineers not only understand how technology works, but how that technology intersects with the people who count on it every day. Today, innovative ideas, insight and new perspectives are at the core of how we create a more powerful, personal and fulfilling experience for all our customers. So, if you’re interested in a career creating breakthrough software and making an impact on an audience of millions, look no further. You won’t just keep up, you’ll break new ground. There are hundreds of opportunities to make your mark on technology and life at American Express. Here’s just some of what you’ll be doing: – Taking your place as a core member of an agile team driving the latest development practices – Writing code and unit tests, working with API specs and automation – Identifying opportunities for adopting new technologies.

Focus:

Designs, develops, solves problems, debugs, evaluates, modifies, deploys, and documents software and systems that meet the needs of customer-facing applications, business applications, and/or internal end user applications.

Organizational Context:

Member of a product engineering or delivery and integration team reporting to a Senior Engineer, Engineering Director or Director Product Delivery & Integration.

How will you make an impact in this role?

Software Development

  • Intermediate to advance level ability in software development, such coding assignments
  • Function as member of an agile team by contributing to software builds through consistent development practices (tools, common components, and documentation)
  • Participate in code reviews and automated testing
  • Debug basic software components and identify code defects for remediation
  • Enable the deployment, support, and monitoring of software across test, integration, and production environments
  • Automate deployments in test or production environments
  • Automatically scale applications based on demand projections

Leadership

  • Demonstrate increased self-reliance to achieve team goals
  • Influence team members with creative changes and improvements by challenging status quo and demonstrating risk taking

Range of Impact/Influence:

  • Accountable for team completing work you’re leading and work you are doing as agreed upon
  • Accountable to team for delivery of quality work

Description:

American Express is looking for energetic, high-performing software engineers to help shape our technology and product roadmap. You will be part of the fast-paced, entrepreneurial team that enables batch/ real-time analytical solutions leveraging transformational technologies (Hadoop/BIG DATA, Data Science – R, Python, Scala, SQL, UNIX , Shell Scripting) to deliver innovative solutions across multiple lines of business’s. Position requires a bachelor’s or higher degree in Computer Science, Information Systems, Engineering or related field. Good experience in data programming and applications systems design and architecture is required.

  • Develops and tests software, including ongoing refactoring of code & drives continuous improvement in code structure & quality
  • Primary focus is spent writing code, API specs, conducting code reviews & testing in ongoing sprints or doing proof of concepts/automation tools
  • Applies visualization and other techniques to fast-track concepts
  • Functions as a core member of an Agile team driving User story analysis & elaboration, design and development of software applications, testing & builds automation tools
  • Works on a specific platform/product or as part of a dynamic resource pool assigned to projects based on demand and business priority
  • Identifies opportunities to adopt innovative technologies
  • Embodies rebel with a cause mentality by looking beyond the obvious for continuous improvement opportunities.
  • Offer of employment with American Express is conditioned upon the successful completion of a background verification check, subject to applicable laws and regulations.

Qualifications:

  • Support the Compliance and Operations Risk big data delivery team in India to lead and assist in the design and actual development of applications.
  • Responsible for specific functional areas within the team, this involves project management and taking business specifications.
  • The individual should be able to independently run projects/tasks delegated to him.
  • Responsible for analysis, design, development, testing, and implementation of system applications.
  • Develop and maintain project related documentation,
  • Provide system support and user support/training/communication for implemented applications.

Technical Skills:

  • Bachelor’s degree in engineering or Computer Science or equivalent OR master’s in computer applications or equivalent
  • 3 to 7 years of experience within Data Engineering/ Data Warehousing using Big Data
  • Expert on Distributed ecosystem including Map-Reduce, Hive, Spark (core, SQL and pyspark)
  • Hands-on experience with programming using Core Java or Python/Scala
  • Expert on Hadoop and Spark Architecture and its working principle 
  • Hands-on experience on writing and understanding complex SQL(Hive/PySpark-dataframes), optimizing joins while processing huge amount of data
  • Experience in UNIX shell scripting
  • Experience with NoSQL i.e., HBase, Couchbase, MongoDB is good to have
  • Ability to design and develop optimized Data pipelines for batch and real time data processing
  • Knowledge of cloud platforms like AWS, GCP is good to have
  • Provides advanced knowledge of technical and functional principles.
  • Should have experience in analysis, design, development, testing, and implementation of system applications
  • Demonstrated ability to develop and document technical and functional specifications and analyze software and system processing flows.
  • Experience with GitHub/Bitbucket and leveraging CI/CD pipelines
  • Creative problem solving (Innovative)
  • Excellent technical and analytical aptitude
  • Self-starter with a curiosity and appetite for new technology,
  • Teamwork & ability to multi-task
  • Excellent communication skills
  • Ability to influence and lead others
  • Ability to coach and mentor,
  • Excellent Project management skills.
  • Results driven
  • Willingness to understand the business and participate in discussions around project requirements

American Express is an equal opportunity employer and makes employment decisions without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, protected veteran status, disability status, age, or any other status protected by law.

Offer of employment with American Express is conditioned upon the successful completion of a background verification check, subject to applicable laws and regulations.