Browse
···
Log in / Register

Java Developer with Web Crawler Experience

Negotiable Salary

Axiom Software Solutions Limited

Austin, TX, USA

Favourites
Share

Description

Role: Java Developer with Web Crawler Experience Location: Austin TX(Hybrid) Responsibilities: 1. Web Crawler Development: Design and implement efficient and scalable web crawlers in Java to collect data from various online sources. 2. Data Extraction: Develop and maintain systems for structured data extraction, handling various data formats (HTML, JSON, XML, etc.). 3. Data Storage and Processing: Design data storage and processing pipelines, ensuring extracted data is clean, structured, and easily accessible. 4. Performance Optimization: Optimize web crawling processes for speed, efficiency, and accuracy, while ensuring minimal impact on source websites. 5. Error Handling and Logging: Implement error-handling mechanisms and logging systems to detect and resolve issues during crawling operations. 6. Data Integrity and Compliance: Ensure data collection practices are ethical, legal, and compliant with relevant regulations (e.g., robots.txt, copyright laws). Requirements: Proficiency in Java and experience with Java-based web scraping libraries (e.g., Jsoup, Apache HttpClient). Knowledge of web crawling frameworks and tools, such as Scrapy, Selenium, or Puppeteer. Strong understanding of HTML, CSS, JavaScript, and web data structures. Familiarity with data parsing and handling techniques for JSON, XML, and other common formats. Experience with database technologies (SQL, NoSQL) to store and manage scraped data. Knowledge of HTTP protocols, headers, proxies, and load handling.

Source:  workable View original post

Location
Austin, TX, USA
Show map

workable

You may also like

Craigslist
Hiring Research Data Analyst! Remote Position! (Lakewood)
Position Summary: The salary range for this position is $67,716.00 to $82,512.00 annually. The Research Data Analyst is an 18-month, non-permanent appointment. This position is approved to work fully remotely. Candidates for this position must live in, or be willing to relocate to, Washington State. This position reports to the Dean of Institutional Research and Effectiveness and requires regular and consistent attendance. The mission of Pierce College is to create quality educational opportunities to a diverse community of learners to thrive in an evolving world. The Pierce College District is committed to being an anti-racist institution; to truly be anti-racist we must work deliberately to eliminate barriers that have prevented the most promising and capable students from entering and thriving in college. We work to ensure that every student in the Pierce College District feels valued, respected, and inspired to achieve their goals. We provide a brave space for Black, Indigenous, Persons of Color (BIPOC), LGBTQ/QTPOC students, students with disabilities, and all students and staff to seek refuge, and an opportunity for them to cultivate their strengths, intelligence, resilience, and creativity. To learn more about our Equity, Engagement, and Belonging efforts, please visit: https://www.pierce.ctc.edu/equity-engagement-and-belonging. The Research Data Analyst is a key member of the Institutional Research team, contributing to the College's data and research efforts. The department supports the college’s mission and commitment to student belonging, learning, and thriving by providing quality, accessible, reliable, and relevant information to facilitate decision-making and planning processes, enhance institutional effectiveness, and support an institutional climate of evidence-based inquiry. Reporting to the Dean of Institutional Research and Effectiveness, the Research Data Analyst collaborates with Business Systems Analysts, Web Services, and other stakeholders to enhance data accessibility within the institution, ensuring accurate and efficient delivery of information. The Research Data Analyst develops interactive data visualizations, dashboards, and reports, that translate complex datasets into actionable information for the institution. To view the full job posting, and to apply, please visit: https://www.schooljobs.com/careers/piercedist/jobs/5051800/research-data-analyst-data-consultant-3?pagetype=transferJobs
9504 Farwest Dr SW, Lakewood, WA 98498, USA
$67,716-82,512/year
Workable
.NET Application Developer
Department of Natural Resources (DNR) is seeking a highly motivated and experienced .NET Lead Application Developer to join DNR App Dev team. This role offers an opportunity to work in a mission-driven environment, supporting Iowa Drinking Water Data Portal and Water Allocation Compliance through a large portfolio of data collection and reporting applications. As a key member of our team, you'll be instrumental in driving our development initiatives and delivering secure, high-quality technical solutions to our customers. You will collaborate closely with teams to optimize development practices and automate pipelines that support large data volumes. This position requires attendance during standard business hours Monday through Friday, with initial on-site presence strongly preferred for orientation and team onboarding. As the .NET Lead Application Developer, you will: •    Have strong communications skills and the ability to work closely with a team. •    Have ability to read, understand, and create design documents. •    Have technical understanding of infrastructure components, their dependencies, and interactions between servers, virtual systems, networks,  databases, web applications, etc. •    Have ability to proactively participate in customer engagements, follow direction from team leads, work with customers, and participate in technical planning. •    Design and implement highly scalable DevOps processes to support the development and deployment of large-scale applications. •    Assess and improve coding and review practices, ensuring a high standard of code quality and security by integrating automated scanning tools into the development cycle. •    Proactively address technical debt before it escalates and ensure continuous improvement of development processes. •    Optimize processes around code branching, merging, and releases to facilitate seamless and efficient deployments.  •    Automate build and deployment processes, reducing manual effort and increasing operational efficiency. •    Enhance unit test and integration test coverage, ensuring application reliability and performance.
Des Moines, IA, USA
Negotiable Salary
Craigslist
Hiring Research Data Analyst! Remote Position! (Lakewood)
Position Summary: The salary range for this position is $67,716.00 to $82,512.00 annually. The Research Data Analyst is an 18-month, non-permanent appointment. This position is approved to work fully remotely. Candidates for this position must live in, or be willing to relocate to, Washington State. This position reports to the Dean of Institutional Research and Effectiveness and requires regular and consistent attendance. The mission of Pierce College is to create quality educational opportunities to a diverse community of learners to thrive in an evolving world. The Pierce College District is committed to being an anti-racist institution; to truly be anti-racist we must work deliberately to eliminate barriers that have prevented the most promising and capable students from entering and thriving in college. We work to ensure that every student in the Pierce College District feels valued, respected, and inspired to achieve their goals. We provide a brave space for Black, Indigenous, Persons of Color (BIPOC), LGBTQ/QTPOC students, students with disabilities, and all students and staff to seek refuge, and an opportunity for them to cultivate their strengths, intelligence, resilience, and creativity. To learn more about our Equity, Engagement, and Belonging efforts, please visit: https://www.pierce.ctc.edu/equity-engagement-and-belonging. The Research Data Analyst is a key member of the Institutional Research team, contributing to the College's data and research efforts. The department supports the college’s mission and commitment to student belonging, learning, and thriving by providing quality, accessible, reliable, and relevant information to facilitate decision-making and planning processes, enhance institutional effectiveness, and support an institutional climate of evidence-based inquiry. Reporting to the Dean of Institutional Research and Effectiveness, the Research Data Analyst collaborates with Business Systems Analysts, Web Services, and other stakeholders to enhance data accessibility within the institution, ensuring accurate and efficient delivery of information. The Research Data Analyst develops interactive data visualizations, dashboards, and reports, that translate complex datasets into actionable information for the institution. To view the full job posting, and to apply, please visit: https://www.schooljobs.com/careers/piercedist/jobs/5051800/research-data-analyst-data-consultant-3?pagetype=transferJobs
9504 Farwest Dr SW, Lakewood, WA 98498, USA
$67,716-82,512/year
Cookie
Cookie Settings
Our Apps
Download
Download on the
APP Store
Download
Get it on
Google Play
© 2025 Servanan International Pte. Ltd.