Browse
···
Log in / Register

Java Developer with Web Crawler Experience

Negotiable Salary

Axiom Software Solutions Limited

Austin, TX, USA

Favourites
Share
Description

Role: Java Developer with Web Crawler Experience Location: Austin TX(Hybrid) Responsibilities: 1. Web Crawler Development: Design and implement efficient and scalable web crawlers in Java to collect data from various online sources. 2. Data Extraction: Develop and maintain systems for structured data extraction, handling various data formats (HTML, JSON, XML, etc.). 3. Data Storage and Processing: Design data storage and processing pipelines, ensuring extracted data is clean, structured, and easily accessible. 4. Performance Optimization: Optimize web crawling processes for speed, efficiency, and accuracy, while ensuring minimal impact on source websites. 5. Error Handling and Logging: Implement error-handling mechanisms and logging systems to detect and resolve issues during crawling operations. 6. Data Integrity and Compliance: Ensure data collection practices are ethical, legal, and compliant with relevant regulations (e.g., robots.txt, copyright laws). Requirements: Proficiency in Java and experience with Java-based web scraping libraries (e.g., Jsoup, Apache HttpClient). Knowledge of web crawling frameworks and tools, such as Scrapy, Selenium, or Puppeteer. Strong understanding of HTML, CSS, JavaScript, and web data structures. Familiarity with data parsing and handling techniques for JSON, XML, and other common formats. Experience with database technologies (SQL, NoSQL) to store and manage scraped data. Knowledge of HTTP protocols, headers, proxies, and load handling.

Source:  workable View original post
Location
Austin, TX, USA
Show map
workable

You may also like

Workable
Dot Net developer
Job Description Dot Net developer with 5-7 years of hands-on development experience . Role involves working directly with clients to understand business requirements and deliver solutions. Collaborating closely with business stakeholders and offshore teams to ensure seamless communication and successful project execution. Requirements: 5-7 years of hands-on experience in DotNet development, with profiency in C#, VB.Net. Strong understanding of the .NET framework, including ASP.NET, ASP.NET Core, and related technologies. Proficiency in database design and SQL, with experience in SQL Server or other database technologies. Experience with web development technologies, including ASP.NET, MVC, APIs, and potentially front-end technologies like HTML, CSS, JavaScript Experience in client-facing roles, with the ability to translate business requirements into technical solutions. Collaborating with business and offshore teams to deliver successful projects. Troubleshoot and resolve complex technical issues in a timely manner. Participate in code reviews, ensuring high standards of quality, performance, and security. Contribute to the architecture and design of software solutions in line with industry best practices. Strong knowledge of software development life cycle, Agile methodologies, and version control systems (e.g., Git). Excellent problem-solving skills and attention to detail. Strong communication skills, with the ability to engage with both technical and non-technical stakeholders. Preferred Skills: Experience with microservices architecture. Knowledge of cloud platforms like AWS, Azure, or Google Cloud. Familiarity with DevOps practices and CI/CD pipelines.
Englewood Cliffs, NJ 07632, USA
Negotiable Salary
About Us
Terms of Use
Privacy Policy
Help
Contact Us
FAQ
Cookie
Cookie Policy
Cookie Settings
Our Apps
Download
Download on the
APP Store
Download
Get it on
Google Play
© 2025 Servanan International Pte. Ltd.