Browse
···
Log in / Register

Java Developer with Web Crawler Experience

Negotiable Salary

Axiom Software Solutions Limited

Austin, TX, USA

Favourites
Share

Description

Role: Java Developer with Web Crawler Experience Location: Austin TX(Hybrid) Responsibilities: 1. Web Crawler Development: Design and implement efficient and scalable web crawlers in Java to collect data from various online sources. 2. Data Extraction: Develop and maintain systems for structured data extraction, handling various data formats (HTML, JSON, XML, etc.). 3. Data Storage and Processing: Design data storage and processing pipelines, ensuring extracted data is clean, structured, and easily accessible. 4. Performance Optimization: Optimize web crawling processes for speed, efficiency, and accuracy, while ensuring minimal impact on source websites. 5. Error Handling and Logging: Implement error-handling mechanisms and logging systems to detect and resolve issues during crawling operations. 6. Data Integrity and Compliance: Ensure data collection practices are ethical, legal, and compliant with relevant regulations (e.g., robots.txt, copyright laws). Requirements: Proficiency in Java and experience with Java-based web scraping libraries (e.g., Jsoup, Apache HttpClient). Knowledge of web crawling frameworks and tools, such as Scrapy, Selenium, or Puppeteer. Strong understanding of HTML, CSS, JavaScript, and web data structures. Familiarity with data parsing and handling techniques for JSON, XML, and other common formats. Experience with database technologies (SQL, NoSQL) to store and manage scraped data. Knowledge of HTTP protocols, headers, proxies, and load handling.

Source:  workable View original post

Location
Austin, TX, USA
Show map

workable

You may also like

Workable
Cloud Software Engineer, Backend
Headquartered in the United States, TP-Link Systems Inc. is a global provider of reliable networking devices and smart home products, consistently ranked as the world’s top provider of Wi-Fi devices. The company is committed to delivering innovative products that enhance people’s lives through faster, more reliable connectivity. With a commitment to excellence, TP-Link serves customers in over 170 countries and continues to grow its global footprint. We believe technology changes the world for the better! At TP-Link Systems Inc, we are committed to crafting dependable, high-performance products to connect users worldwide with the wonders of technology.  Embracing professionalism, innovation, excellence, and simplicity, we aim to assist our clients in achieving remarkable global performance and enable consumers to enjoy a seamless, effortless lifestyle.  Overview: Do you thrive in building robust, scalable backend systems for the cloud? Are you passionate about designing and implementing high-performance, secure applications? If so, then this Cloud Software Engineer, Backend role might be perfect for you! In this role, you will be an important member of our engineering team, responsible for the design, development, and maintenance of our backend cloud applications. You'll collaborate with our Cloud Architects, UX/UI designers, QA teams, and key hardware personnel and you will leverage your expertise in cloud technologies and backend development to build secure, reliable, and performant systems that meet our business needs. Key Responsibilities: Research competitors, design, develop, and maintain highly scalable and reliable backend services using cloud-native technologies. Collaborate with frontend engineers and other teams to ensure seamless integration. Select and utilize appropriate cloud technologies (e.g., AWS, Azure, OCI) to build and deploy applications. Optimize backend systems for performance, scalability, and cost-efficiency. Implement robust security measures to ensure data protection and application integrity. Develop various software (e.g., Windows/Linux/Hardware Box) with the same code which is in DDD architecture. Write clean, maintainable, and well-documented code. Automate infrastructure provisioning and deployment using tools like CI/CD pipelines. Troubleshoot and debug complex backend issues. Participate in code reviews & design reviews that are related to the modules that you are responsible for. Stay up to date on the latest cloud technologies and best practices. Requirements Bachelor's degree in Computer Science, Software Engineering, or a related field. 2+ years of experience in cloud backend software development. Proven experience designing and building scalable, reliable, and secure cloud-based applications. Strong proficiency in backend programming language Java and frameworks like SpringBoot, Service Mesh. Strong proficiency in JVM, multithreading programming, performance optimization skills. Experience with a specific cloud platform (AWS, Azure, OCI). Experience with cloud-based databases (e.g., NoSQL databases such as MongoDB, relational databases). Experience with distributed middleware (e.g. Message queue, config center). Experience with containerization technologies (e.g., Docker, Kubernetes). In-depth knowledge of backend technologies (e.g., APIs, message queues). Experience with security best practices for cloud environments. Excellent communication, collaboration, and problem-solving skills. Ability to work effectively in a team environment. Preferred Qualifications: Experience with DevOps principles and practices. Experience with serverless architectures. Experience with SDN Software Controller Platform. Experience with AI Ops. Benefits Salary range: $120,000 - $180,000 Free snacks and drinks, and provided lunch on Fridays Fully paid medical, dental, and vision insurance (partial coverage for dependents) Contributions to 401k funds Bi-annual reviews, and annual pay increases Health and wellness benefits, including free gym membership Quarterly team-building events At TP-Link Systems Inc., we are continually searching for ambitious individuals who are passionate about their work. We believe that diversity fuels innovation, collaboration, and drives our entrepreneurial spirit. As a global company, we highly value diverse perspectives and are committed to cultivating an environment where all voices are heard, respected, and valued. We are dedicated to providing equal employment opportunities to all employees and applicants, and we prohibit discrimination and harassment of any kind based on race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state, or local laws. Beyond compliance, we strive to create a supportive and growth-oriented workplace for everyone. If you share our passion and connection to this mission, we welcome you to apply and join us in building a vibrant and inclusive team at TP-Link Systems Inc. Please, no third-party agency inquiries, and we are unable to offer visa sponsorships at this time.
Irvine, CA, USA
$120,000-180,000/year
Workable
Java Developer
Role: Java Developer Location: Philadelphia, Philadelphia, PA Mandatory skills: Java, Spring, Jenkins, Tomcat, Python, MySQL, GIT, Test Automation Framework Development Bachelor's degree (or foreign equivalent) in Computer Science, Engineering, or a related technical field Define and develop test automation using the following: Java J2EE, Spring, Jenkins, Tomcat, Python, MySQL, GIT and Maven Deploy using Agile or Scrum in a DevOps environment. Additional advantage having +1 year of experience or domain in developing in video and broadband product architecture utilizing RDK/TDK suite. Key Job Responsibility: +5 years DevOps Practices: •Full Lifecycle Ownership: Oversee solutions through their entire lifecycle, from design to production deployment. •Comprehensive Solutioning: Design, develop, execute, train, operationalize, monitor, and triage end-to-end solutions. Focus on enhancing process quality and predictability. •Observability and Oversight: Improve observability to ensure comprehensive oversight of the end-to-end release readiness process. •Process Optimization: Develop and implement process improvements to boost team velocity and efficiency. •Collaboration and Integration: Work closely with other engineers, integrating business and functional priorities into solutions. Act as a key contributor in complex, high-stakes environments. •Operationalization: Successfully operationalize solutions in production to ensure seamless functionality and impact. 2.+5 years BDD and Test Automation Expertise: •Have a good understanding of Business-Driven Development strategy. •Explore advanced topics such as test data management, scenario refactoring, and integrating BDD with continuous integration/continuous deployment (CI/CD) pipelines. •Develop, and execute high-quality test plans efficiently on RDK stacks while leveraging automation and tooling to detect defects quickly and ensure the reliability of test coverages. 3. +5 years Tooling and Test Automation Infrastructure: •Get familiar with automation tools and frameworks for test development, execution, and reporting. •Maintaining test automation, including selecting appropriate tools, configuring environments, and integrating with version control systems. 4. +5 years Reliability Engineering and Process Optimization: •Explore strategies for optimizing validation coverage for home setups, standalone. •Practice DevOps to reduce the time between testbed delivery and acceptance by the receiving teams with reliability metrics 5. Collaboration and Communication Skills: •Effectively communicate and collaborate within the team and with stakeholders. •Practice agile methodologies to enhance teamwork and productivity. •Participate in cross-functional activities such as joint sprint planning, backlog grooming, and retrospective meetings to foster alignment and shared understanding across teams. 6. Continuous Learning and Improvement: •Practice continuous learning and improvement for ongoing skill development and career growth. •Stay updated on industry trends and best practices. •Put the team first. Value individual growth and collective success. 7. Own the customer experience - think and act in ways that put our customers first, give them seamless digital options at every touchpoint, and make them promoters of our products and services. Know your stuff - be enthusiastic learners, users and advocates of our game-changing technology, products, and services, especially our digital tools and experiences. 8. Win as a team - make big things happen by working together and being open to new ideas. Be an active part of the Net Promoter System - a way of working that brings more employee and customer feedback into the company - by joining huddles, making call backs, and helping us elevate opportunities to do better for our customers. Drive results and growth. Respect and promote inclusion & diversity. Do what's right for each other, our customers, investors, and our communities.
Philadelphia, PA, USA
Negotiable Salary
Workable
Data Engineer (USA)
Trexquant is a growing systematic fund manager with a core team of highly accomplished technologists. We apply a wide variety of statistical and machine learning techniques to build investment portfolios and trade our client assets in global equity and futures markets. With locations in the US, China and India, our global team in excess of 50 employees is comprised primarily of research professionals with advanced science, math and technology degrees who explore the universe of quantitative methods for opportunities to enhance and adapt our platform and profit in an exciting and dynamic environment. We are seeking a dedicated and detail-oriented Data Engineer to join our team. The primary goal of this team is to assist researchers in writing, downloading and reading scripts, effectively translating their ideas into data variables for trading purposes. The ideal candidate will have a strong background in data engineering, excellent scripting skills, and a keen understanding of financial data and trading strategies. Responsibilities Collaborate with researchers to understand their data requirements and trading strategies. Develop and maintain scripts for downloading, reading, and processing data from various sources. Ensure data accuracy, consistency, and reliability by implementing quality control measures. Optimize data retrieval processes for efficiency and performance. Assist in the integration of new data sources and formats into the existing data infrastructure. Provide technical support and troubleshooting for data-related issues. Document data workflows, processes, and scripts for future reference and knowledge sharing. Stay updated with industry trends and advancements in data engineering and financial data. Requirements A degree in a technical discipline (computer science, mathematics, statistics, physics, etc.)  1+ years of experience in Python as used in data capacity Experience in financial services and working with financial data providers is a plus Ability to work independently and take projects to completion, quickly learn new systems, think creatively and pay attention to details Benefits Competitive salary plus bonus bonus based on individual and company performance Collaborative, Casual, and friendly work environment PPO Health, dental and vision insurance premiums fully covered for you and your dependents Pre-tax commuter benefits Weekly company meals Trexquant is an Equal Opportunity Employer
Stamford, CT, USA
Negotiable Salary
Workable
Senior/Lead Backend (NodeJS) Engineer - Onsite
About Deep Origin Led by Michael Antonov, a co-founder of Oculus, and well-funded by Formic Ventures, Deep Origin is poised to reinvent the way scientists work and life science innovations come to life. We see a future largely free of diseases, with a 150-year lifespan being a norm. To get there, we are building an operating system for science, enabling scientists to be more productive and to bring tomorrow's ideas to life quickly and at a reasonable cost. Applicants must be authorized to work for any employer in the U.S. We are unable to sponsor or take over sponsorship of an employment Visa at this time. Role Description In this hands-on position, you will be a key member of the software engineering team, building our key functionality and integrating with key partners. Your responsibilities will range from designing and developing complex, large-scale systems to writing APIs that integrate with various cloud providers and partners. You will have ownership in key software feature areas and their architectural design, as well as software implementation with a high level of independence and impact. Requirements 7+ years of experience designing, building, and operating complex, highly-scalable, distributed applications and systems 3+ years of hands-on software development experience with TypeScript/JavaScript/NodeJS Experience with both relational databases (e.g. Postgres) and NOSQL (e.g. MongoDB) Knowledge of Kubernetes and Cloud infrastructure/deployment tools (specifically with cluster operations and operators) Has built platforms from an early stage Has scaled platforms to handle many users (10,000+ DAU) Has extensive system-design experience Has experience designing systems with complex data-sets/relations Has experience working with distributed systems/platforms Thinks about architecture first and how the code fits in second Has experience working with/implementing a multi-tenant system Systematic problem-solving approach, coupled with a strong sense of ownership and drive Ability to work both independently and on the team Experience working in high-energy startups with fast product delivery mechanisms Benefits Benefits This position offers a competitive salary, benefits, and equity.
South San Francisco, CA, USA
Negotiable Salary
Workable
Software Dev Engineer IV - Python Developer
Position: Software Dev Engineer IV  Location: New York, NY, 10018  Duration: 6 Months       Job Type: Contract         Work Type:  Hybrid           Job Description:     Design, develop, implement, test, document and deploy full-stack, cloud-native, contact center-related software applications, tools, systems and services using multi-threaded programming, development in Python and React/node.js, implementing architecture patterns and design patterns, and utilizing generative AI large language models. Assist in gathering and analyzing business and functional requirements, and translate requirements into technical specifications for robust, scalable, supportable solutions that work well within the overall system architecture. Own delivery of entire piece of system or application, and serve as technical lead on complex projects using best practice engineering standards. Produce comprehensive, usable software documentation.  Qualifications:  MS or BS in Computer Science, Computer or Electrical Engineering, Mathematics, or a related field, plus five years of progressively responsible experience in the job offered or related occupations of Software Engineer, Software Developer, or related.  Required technical skills:  Coding proficiency in Python, and front-end development experience with Javascript/React.  Proficiency development with services such as AWS Lambda, Step Functions, DynamoDB, AppSync, Bedrock, SageMaker, and CloudWatch.  Proficiency in developing and integrating with REST-based or GraphQL-based APIs.  Proficiency in developing infrastructure-as-code deployment solutions such as AWS CloudFormation or AWS CDK .  Experience collaborating with other developers using git repositories, including creating and managing feature branches, pull requests, code merge, and GitHib actions or equivalent.  Preferred skills:  Experience with Contact Center development and telephony infrastructure.  Experience with prompt engineering for modern large language models.  Experience using modern AI-based agentic coding assistants for code development, test development, and documentation.  Track record of building successful serverless architectures following AWS Well Architected principles.  Candidate Requirements:  REQUIRED SKILLS:  Please refer required and technical skills in the job description that is what the manager needs on candidates resumes   Years of Experience: 5+   Degree or Certification:  Bachelors’ degree preferred  Top 3 must-have hard skills  Generative AI based coding  AWS serverless    Python and JavaScript/React 
New York, NY, USA
Negotiable Salary
Workable
Front End Engineer
About Us Resonance is building a more sustainable and valuable fashion industry for designers, brands, manufacturers, consumers, and the planet. The company’s AI-powered operating system ONE enables Brands to Design, Sell and Make in that order, empowering designers to operate with no unnecessary inventory—eliminating the financial and environmental burdens of the legacy fashion industry. Headquartered in New York City and Santiago, Dominican Republic, Resonance has partnered with more than 30 brands, including THE KIT and Rebecca Minkoff to create garments that use 97% less dye, 70% less water, and 50% less material than any other fashion brand—and immediately eliminate overproduction. Want to know more? Visit our website and read articles about us. About the Role We’re looking to add a front-end software engineer to our team to report directly to our technical cofounder and CEO, Christian Gheorghe and rapidly build interactive user experiences and interfaces for our brands designing, selling and operating on our self-sufficient digital product ONE. This role engages across product management, platform engineering and design. Individuals in this role supercharge our ability to rapidly ship new experiences and learn from live users. Responsibilities include: Build best in class user experiences for highly visual, creative users across a myriad of functions including: 3D design, e-commerce/channel selling systems, planning tools, and more. Scope, design and lead implementation of technical projects, laying groundwork for early-stage products to iteratively evolve and scale. Ensure UI components and libraries are reliable, secure and accessible. Independently and collaboratively solve UI/UX design problems, including debugging for long term stability. You may be a good fit if the following describes you: You’re a coder, a hacker, an engineer at heart. You like to create something from nothing. You appreciate beautiful, highly opinionated, custom built, user experiences and design systems. You are intensely curious and desire a role where you collaborate with creators, designers and manufacturers. You have love for analog and digital products, from vinyl LPS to beautiful VR applications. Requirements Job Requirements 3+ years of experience developing design systems. Strong foundation in front-end frameworks, design patterns, data structures, storage systems and web services. Proficient in web technologies: React, JavaScript, HTML, CSS, modern package bundlers/task runners and testing tools. Experience with CI/CD pipelines and how to integrate frontend applications in that pipeline. Experience with Webpack, Storybook, GraphQL, and AWS a plus. Able to independently solve UI/UX design problems. Computer Science Background Comfortable with being given clearly-defined goals but full ownership and responsibility for discovering the path to achieve those goals. Ability to succeed in a constantly changing, fast-paced environment. Making mistakes means we are learning, and you welcome actively seeking ways to grow Benefits We offer full benefits (medical, dental, and vision), a competitive salary and equity Resonance Companies is an equal opportunity employer and values diversity in our company. We do not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, non-disqualifying physical or mental disability, national origin, veteran status, or any other basis covered by appropriate law. All employment is decided on the basis of qualifications, merit, and business need.
New York, NY, USA
Negotiable Salary
Cookie
Cookie Settings
Our Apps
Download
Download on the
APP Store
Download
Get it on
Google Play
© 2025 Servanan International Pte. Ltd.