Browse
···
Log in / Register

Java Developer with Web Crawler Experience

Negotiable Salary

Axiom Software Solutions Limited

Austin, TX, USA

Favourites
Share

Description

Role: Java Developer with Web Crawler Experience Location: Austin TX(Hybrid) Responsibilities: 1. Web Crawler Development: Design and implement efficient and scalable web crawlers in Java to collect data from various online sources. 2. Data Extraction: Develop and maintain systems for structured data extraction, handling various data formats (HTML, JSON, XML, etc.). 3. Data Storage and Processing: Design data storage and processing pipelines, ensuring extracted data is clean, structured, and easily accessible. 4. Performance Optimization: Optimize web crawling processes for speed, efficiency, and accuracy, while ensuring minimal impact on source websites. 5. Error Handling and Logging: Implement error-handling mechanisms and logging systems to detect and resolve issues during crawling operations. 6. Data Integrity and Compliance: Ensure data collection practices are ethical, legal, and compliant with relevant regulations (e.g., robots.txt, copyright laws). Requirements: Proficiency in Java and experience with Java-based web scraping libraries (e.g., Jsoup, Apache HttpClient). Knowledge of web crawling frameworks and tools, such as Scrapy, Selenium, or Puppeteer. Strong understanding of HTML, CSS, JavaScript, and web data structures. Familiarity with data parsing and handling techniques for JSON, XML, and other common formats. Experience with database technologies (SQL, NoSQL) to store and manage scraped data. Knowledge of HTTP protocols, headers, proxies, and load handling.

Source:  workable View original post

Location
Austin, TX, USA
Show map

workable

You may also like

Workable
LDAP Software Engineer - Peru, IN -Fulltime Role
LDAP Software Engineer Location: Peru, IN Duration: Permanent role Responsibilities: * Work with upstream data source and downstream clients to resolve data related issues. * Assist in designing, developing, and implementing Data as A Service (DaaS) and support for microservices. * Perform data and log analysis. * Assist in designing, developing, and implementing flexible LDAP architecture for PaaS and SaaS applications. * Performance periodic maintenance as/when required. Requirements: * Performance periodic maintenance as/when required. * Assist in designing, developing, and implementing flexible LDAP architecture for PaaS and SaaS applications. * Work with upstream data source and downstream clients to resolve data related issues. * Assist in designing, developing, and implementing Data as A Service (DaaS) and support for microservices. * Perform data and log analysis. * Advanced knowledge of LDAP directories based on OpenDJ, Oracle (sun) or ForgeRock directories. * Expert in maintaining, troubleshooting issues in production environment. * Experience in installing and configuring Oracle DSEE 11g & OpenDJ, ForgeRock DS. Required Software Skills: * Directory Services (LDAP, MS Active Directory, MS Entra ID), including Group Policy, DNS, and DHCP Exchange / M365 * VMware vSphere/vCenter * Server and PC operating systems including Windows OS's and Linux * Citrix DaaS, and related components * Microsoft SQL Server and/or Oracle SQL along with database management utilities Recommended Working Knowledge * PowerShell * Git (GitHub / Gitlab) * TCP/IP * Networking OSI mode
Indiana, Peru
Negotiable Salary
Cookie
Cookie Settings
Our Apps
Download
Download on the
APP Store
Download
Get it on
Google Play
© 2025 Servanan International Pte. Ltd.