Browse
···
Log in / Register

Java Developer with Web Crawler Experience

Negotiable Salary

Axiom Software Solutions Limited

Austin, TX, USA

Favourites
Share

Description

Role: Java Developer with Web Crawler Experience Location: Austin TX(Hybrid) Responsibilities: 1. Web Crawler Development: Design and implement efficient and scalable web crawlers in Java to collect data from various online sources. 2. Data Extraction: Develop and maintain systems for structured data extraction, handling various data formats (HTML, JSON, XML, etc.). 3. Data Storage and Processing: Design data storage and processing pipelines, ensuring extracted data is clean, structured, and easily accessible. 4. Performance Optimization: Optimize web crawling processes for speed, efficiency, and accuracy, while ensuring minimal impact on source websites. 5. Error Handling and Logging: Implement error-handling mechanisms and logging systems to detect and resolve issues during crawling operations. 6. Data Integrity and Compliance: Ensure data collection practices are ethical, legal, and compliant with relevant regulations (e.g., robots.txt, copyright laws). Requirements: Proficiency in Java and experience with Java-based web scraping libraries (e.g., Jsoup, Apache HttpClient). Knowledge of web crawling frameworks and tools, such as Scrapy, Selenium, or Puppeteer. Strong understanding of HTML, CSS, JavaScript, and web data structures. Familiarity with data parsing and handling techniques for JSON, XML, and other common formats. Experience with database technologies (SQL, NoSQL) to store and manage scraped data. Knowledge of HTTP protocols, headers, proxies, and load handling.

Source:  workable View original post

Location
Austin, TX, USA
Show map

workable

You may also like

Craigslist
Website Tester - Join Fast
APPLY HERE Join our team of website testers at Little Wheel. We are a gambling technology company researching and building products that put players first. We are hiring Website Testers at all of our offices across North America. Earn at least $1,000 for around 20 hours of testing. You’ll earn a minimum of $25/hour, but faster testers earn more. This is a short-term contract. There are opportunities for ongoing work for high performers. It’s ideally suited to people looking to supplement their income. You will not be gambling with your money, and you do not need any experience. There are no fees or out-of-pocket expenses. You'll complete a short online training program before starting to ensure you have the right skills for the role. We offer a $100 bonus for completing onboarding with us, so you start earning straight away. What's in it for you: Easy money - you will earn at least $1,000 for around 20 hours of testing. Work from home - earn from the comfort of your own home. Flexible schedule - you can choose when you work, day or night. Paid quickly - the testing is split into 4 stages, and you get paid after each. Great support - our research coordinators are available from 6am to 11pm ET. What you need to get started: Be at least 21 years old. Have your own laptop or desktop. What team members say about us: Great side gig for some extra cash. The team is very communicative and will answer any questions/concerns! (Maria, Trustpilot, 2021) Tasks to complete are clearly explained with step by step instructions. There are timelines to complete tasks but you can work on them when convenient for you, usually a few hours each week is all you need. Support is very helpful and responsive. This is an easy and fun way to make some extra money. (Anonymous, Glassdoor, 2025) APPLY NOW
HGHW+C6 Marquette, MI, USA
$25/hour
Workable
Data & BI Senior Data Engineer
Job Description: We are seeking a highly skilled and experienced Senior Data Engineer to join our team. The ideal candidate will have a strong background in data engineering, with a specialization in Matillion, SSIS, Azure DevOps, and ETL processes. This role will involve designing, developing, testing, and deploying ETL jobs, collaborating with cross-functional teams, and ensuring efficient data processing. Key Responsibilities: Design, develop, test, and deploy Matillion ETL jobs in accordance with project requirements. Collaborate with the Data and BI team to understand data integration needs and translate them into Matillion ETL solutions. Create and modify Python code/components in Matillion jobs. Identify opportunities for performance optimization and implement enhancements to ensure efficient data processing. Collaborate with cross-functional teams, including database administrators, data engineers, and business analysts, to ensure seamless integration of ETL processes. Create and maintain comprehensive documentation for Matillion ETL jobs, ensuring knowledge transfer within the team. Create, test, and deploy SQL Server Integration Service (SSIS) packages and schedule them via Active Batch scheduling tool. Create Matillion deployment builds using Azure DevOps CI/CD pipeline and perform release manager activities. Review code of other developers (L2, L3-BI/DI) to ensure code standards and provide approval as part of code review activities. Resolve escalation tickets from the L2 team as part of the on-call schedule. Working knowledge of API and Postman tool is an added advantage. Qualifications: Bachelor's or Master's degree in Computer Science, Engineering, or a related field. 5+ years of experience in data engineering, with a focus on ETL processes. Proficiency in Matillion, SSIS, Azure DevOps, and ETL. Strong knowledge of SQL, Python, and data integration techniques. Experience with performance optimization and data processing enhancements. Excellent collaboration and communication skills. Ability to work in a fast-paced, dynamic environment. Preferred Skills: Experience with cloud platforms such as AWS or Azure. Knowledge of data warehousing and data modeling. Familiarity with DevOps practices and CI/CD pipelines. Strong problem-solving skills and attention to detail.
Atlanta, GA, USA
Negotiable Salary
Craigslist
Contract Senior Software Engineer (Remote) (Vista, CA)
Location: Remote (occasional meetings in Vista, CA) Type: Contract (3–6 months, potential for long-term) Hours: 20–40 hours/week Compensation: Hourly rate commensurate with experience, paid weekly About Us We are an established software development company in San Diego North County with a strong track record of delivering robust solutions across desktop, web, and mobile platforms. We're seeking a Senior Software Engineer to join us on a contract basis to support several ongoing and upcoming projects. This role is nearly 100% remote, with only rare in-person meetings in Vista, CA. What You’ll Work On You’ll contribute to a variety of projects, including modernizing legacy applications, building new features, and supporting mobile development efforts. Your work will span across desktop, server, and mobile technologies. Required Skills We’re looking for candidates with strong experience in the following areas: • Microsoft C#: Windows Forms development (UI and back-end logic) • SQL Server: Refactoring inline SQL into stored procedures • VB6: Ability to read and understand legacy code (for partial porting) • Python: General scripting and Odoo customization • Mobile Development: • Objective-C/Xcode: For potential modernization of the FABRIC app • MAUI: For converting iOS apps into multi-platform solutions • Expo / React / React Native How to Apply Please send your current resume and contact information to be considered. We will conduct phone interviews with top candidates.
1145 Oak Dr, Vista, CA 92084, USA
Negotiable Salary
Cookie
Cookie Settings
Our Apps
Download
Download on the
APP Store
Download
Get it on
Google Play
© 2025 Servanan International Pte. Ltd.