Browse
···
Log in / Register

Java Developer with Web Crawler Experience

Negotiable Salary

Axiom Software Solutions Limited

Austin, TX, USA

Favourites
Share

Description

Role: Java Developer with Web Crawler Experience Location: Austin TX(Hybrid) Responsibilities: 1. Web Crawler Development: Design and implement efficient and scalable web crawlers in Java to collect data from various online sources. 2. Data Extraction: Develop and maintain systems for structured data extraction, handling various data formats (HTML, JSON, XML, etc.). 3. Data Storage and Processing: Design data storage and processing pipelines, ensuring extracted data is clean, structured, and easily accessible. 4. Performance Optimization: Optimize web crawling processes for speed, efficiency, and accuracy, while ensuring minimal impact on source websites. 5. Error Handling and Logging: Implement error-handling mechanisms and logging systems to detect and resolve issues during crawling operations. 6. Data Integrity and Compliance: Ensure data collection practices are ethical, legal, and compliant with relevant regulations (e.g., robots.txt, copyright laws). Requirements: Proficiency in Java and experience with Java-based web scraping libraries (e.g., Jsoup, Apache HttpClient). Knowledge of web crawling frameworks and tools, such as Scrapy, Selenium, or Puppeteer. Strong understanding of HTML, CSS, JavaScript, and web data structures. Familiarity with data parsing and handling techniques for JSON, XML, and other common formats. Experience with database technologies (SQL, NoSQL) to store and manage scraped data. Knowledge of HTTP protocols, headers, proxies, and load handling.

Source:  workable View original post

Location
Austin, TX, USA
Show map

workable

You may also like

Workable
Software Dev Engineer IV
Job Title: Software Dev Engineer IV Location: Herndon, VA, 20171 Duration: 6 Months       Job Type: Contract         Work Type: Hybrid   Job Description:  Design, develop, implement, test, document and deploy full-stack, cloud-native, contact center-related software applications, tools, systems and services using multi-threaded programming, development in Python and React/node.js, implementing architecture patterns and design patterns, and utilizing generative AI large language models. Assist in gathering and analyzing business and functional requirements, and translate requirements into technical specifications for robust, scalable, supportable solutions that work well within the overall system architecture. Own delivery of entire piece of system or application, and serve as technical lead on complex projects using best practice engineering standards. Produce comprehensive, usable software documentation. Qualifications: MS or BS in Computer Science, Computer or Electrical Engineering, Mathematics, or a related field, plus five years of progressively responsible experience in the job offered or related occupations of Software Engineer, Software Developer, or related. Required technical skills: Coding proficiency in Python, and front-end development experience with Javascript/React. Proficiency development with services such as AWS Lambda, Step Functions, DynamoDB, AppSync, Bedrock, SageMaker, and CloudWatch. Proficiency in developing and integrating with REST-based or GraphQL-based APIs. Proficiency in developing infrastructure-as-code deployment solutions such as AWS CloudFormation or AWS CDK . Experience collaborating with other developers using git repositories, including creating and managing feature branches, pull requests, code merge, and GitHib actions or equivalent. Preferred skills: Experience with Contact Center development and telephony infrastructure. Experience with prompt engineering for modern large language models. Experience using modern AI-based agentic coding assistants for code development, test development, and documentation. Track record of building successful serverless architectures following AWS Well Architected principles. Candidate Requirements:  Years of Experience: 5+ Years Degree or Certification: Bachelors’ degree preferred Top 3 must-have hard skills:  Generative AI based coding AWS serverless Python and JavaScript/React Required: 5+ years-* Python Developer,  Javascript/React, AWS, (Gen AI/ AI / ML/ MLops / etc..,) Only Hybrid 
Herndon, VA 20170, USA
Negotiable Salary
Cookie
Cookie Settings
Our Apps
Download
Download on the
APP Store
Download
Get it on
Google Play
© 2025 Servanan International Pte. Ltd.