Browse
···
Log in / Register

Java Developer with Web Crawler Experience

Negotiable Salary

Axiom Software Solutions Limited

Austin, TX, USA

Favourites
Share

Description

Role: Java Developer with Web Crawler Experience Location: Austin TX(Hybrid) Responsibilities: 1. Web Crawler Development: Design and implement efficient and scalable web crawlers in Java to collect data from various online sources. 2. Data Extraction: Develop and maintain systems for structured data extraction, handling various data formats (HTML, JSON, XML, etc.). 3. Data Storage and Processing: Design data storage and processing pipelines, ensuring extracted data is clean, structured, and easily accessible. 4. Performance Optimization: Optimize web crawling processes for speed, efficiency, and accuracy, while ensuring minimal impact on source websites. 5. Error Handling and Logging: Implement error-handling mechanisms and logging systems to detect and resolve issues during crawling operations. 6. Data Integrity and Compliance: Ensure data collection practices are ethical, legal, and compliant with relevant regulations (e.g., robots.txt, copyright laws). Requirements: Proficiency in Java and experience with Java-based web scraping libraries (e.g., Jsoup, Apache HttpClient). Knowledge of web crawling frameworks and tools, such as Scrapy, Selenium, or Puppeteer. Strong understanding of HTML, CSS, JavaScript, and web data structures. Familiarity with data parsing and handling techniques for JSON, XML, and other common formats. Experience with database technologies (SQL, NoSQL) to store and manage scraped data. Knowledge of HTTP protocols, headers, proxies, and load handling.

Source:  workable View original post

Location
Austin, TX, USA
Show map

workable

You may also like

Workable
Cloud Engineer - Java Backend Developer
Headquartered in the United States, TP-Link Systems Inc. is a global provider of reliable networking devices and smart home products, consistently ranked as the world’s top provider of Wi-Fi devices. The company is committed to delivering innovative products that enhance people’s lives through faster, more reliable connectivity. With a commitment to excellence, TP-Link serves customers in over 170 countries and continues to grow its global footprint. We believe technology changes the world for the better! At TP-Link Systems Inc, we are committed to crafting dependable, high-performance products to connect users worldwide with the wonders of technology.  Embracing professionalism, innovation, excellence, and simplicity, we aim to assist our clients in achieving remarkable global performance and enable consumers to enjoy a seamless, effortless lifestyle. Responsibilities: Design and develop the cloud service architecture for the unified management platform of telecom operator equipment. Architect and develop account, user, payment, and analytics systems for the operator management platform. Participate in software development processes, including requirements analysis, architecture design, coding, and testing. Contribute to the successful delivery of small to medium-sized projects. Conduct research on cutting-edge cloud technologies and explore new business scenarios. Requirements Educational Background Bachelor's degree or higher in Computer Science, Software Engineering, or a related field; a Master’s degree is preferred. Work Experience At least 2 years of experience in designing highly available, high-concurrency, and highperformance distributed architectures. Extensive experience in designing and implementing architectures for large-scale internet platforms or enterprise systems. Professional Skills Strong foundation in Java, with in-depth understanding of JVM internals. Hands-on experience with commonly used middleware technologies such as Redis, Kafka, and gRPC. Proficiency in frameworks and technologies such as SpringMVC, Netty, Spring Cloud, and Service Mesh. Familiarity with the design and development of major relational and NoSQL databases like MySQL, Cassandra, and MongoDB. Expertise in design patterns, with strong coding best practices and documentation skills. Additional Skills (Preferred, but Not Required): Experience in designing and implementing architectures on public cloud platforms (e.g., AWS, Azure, or GCP). Experience in large-scale data processing Benefits Salary range: $100,000-140,000 Free snacks and drinks, and provided lunch on Fridays Fully paid medical, dental, and vision insurance (partial coverage for dependents) Contributions to 401k funds Bi-annual reviews, and annual pay increases Health and wellness benefits, including free gym membership Quarterly team-building events At TP-Link Systems Inc., we are continually searching for ambitious individuals who are passionate about their work. We believe that diversity fuels innovation, collaboration, and drives our entrepreneurial spirit. As a global company, we highly value diverse perspectives and are committed to cultivating an environment where all voices are heard, respected, and valued. We are dedicated to providing equal employment opportunities to all employees and applicants, and we prohibit discrimination and harassment of any kind based on race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state, or local laws. Beyond compliance, we strive to create a supportive and growth-oriented workplace for everyone. If you share our passion and connection to this mission, we welcome you to apply and join us in building a vibrant and inclusive team at TP-Link Systems Inc. Please, no third-party agency inquiries, and we are unable to offer visa sponsorships at this time.
Irvine, CA, USA
$100,000-140,000/year
Workable
Oracle IDM /ADF - Washington, DC (Long-Term)
Title: Oracle IDM (Oracle Identity Management) Location: Washington, DC Position: Contract Duration: 1+ Year’s Rate: $Market/Hr Job Overview & Responsibilities:- The candidate will provide technical consulting/advisory, implementation, as well as general maintenance and operational support services to our clients in the following areas: • Experience with Oracle IDM (Oracle Identity Management) OR Oracle ADF. • Experience with Sailpoint Identity IQ, including design, development and implementation skills. • Experience in implementation of enterprise-level, and distributed server-side applications Integration experience with database server technologies such as Microsoft SQL, Oracle, etc. • Integration experience with web server technologies such as Apache, IIS, etc. • Integration experience with directory server technologies such as Active Directory, openLdap, etc. • Ability to meet project schedule deliverables • Demonstrated ability to write documentation deliverables including recommendations, assessments, root cause analyses, project roadmaps, and other reports • Must possess excellent Microsoft Word, Excel, Visio, and PowerPoint skills • Security product certifications desired; CISSP • General Web development and programming skills with either Java, .NET, PHP, HTML, XML, CSS, Perl, shell, or SQL are highly desirable • Experience with smart card technology, Smart Card Management Systems, and PKI a plus • Must be familiar with Federal policies and standards on information security and authentication (FIPS, NIST, PIV, etc.) • Solid understanding of Federal credentialing standards (PIV, PIV-I/C, TWIC, FRAC, etc.) • Experience in assessment, evaluation, and design of solutions related to encryption and key management, identity management, strong authentication, and end-point security • Experience in implementation of federation/SAML technologies such as ADFS, etc. Requirements Note: If interested please send your updated resume to gowri.sankar@two95intl.com and include your Salary requirement along with your contact details with a suitable time when we can reach you. If you know of anyone in your sphere of contacts, who would be a perfect match for this job then, we would appreciate if you can forward this posting to them with a copy to us. We look forward to hearing from you at the earliest!
Washington, DC, USA
Negotiable Salary
Workable
Data Engineer (USA)
Trexquant is a growing systematic fund manager with a core team of highly accomplished technologists. We apply a wide variety of statistical and machine learning techniques to build investment portfolios and trade our client assets in global equity and futures markets. With locations in the US, China and India, our global team in excess of 50 employees is comprised primarily of research professionals with advanced science, math and technology degrees who explore the universe of quantitative methods for opportunities to enhance and adapt our platform and profit in an exciting and dynamic environment. We are seeking a dedicated and detail-oriented Data Engineer to join our team. The primary goal of this team is to assist researchers in writing, downloading and reading scripts, effectively translating their ideas into data variables for trading purposes. The ideal candidate will have a strong background in data engineering, excellent scripting skills, and a keen understanding of financial data and trading strategies. Responsibilities Collaborate with researchers to understand their data requirements and trading strategies. Develop and maintain scripts for downloading, reading, and processing data from various sources. Ensure data accuracy, consistency, and reliability by implementing quality control measures. Optimize data retrieval processes for efficiency and performance. Assist in the integration of new data sources and formats into the existing data infrastructure. Provide technical support and troubleshooting for data-related issues. Document data workflows, processes, and scripts for future reference and knowledge sharing. Stay updated with industry trends and advancements in data engineering and financial data. Requirements A degree in a technical discipline (computer science, mathematics, statistics, physics, etc.)  1+ years of experience in Python as used in data capacity Experience in financial services and working with financial data providers is a plus Ability to work independently and take projects to completion, quickly learn new systems, think creatively and pay attention to details Benefits Competitive salary plus bonus bonus based on individual and company performance Collaborative, Casual, and friendly work environment PPO Health, dental and vision insurance premiums fully covered for you and your dependents Pre-tax commuter benefits Weekly company meals Trexquant is an Equal Opportunity Employer
Stamford, CT, USA
Negotiable Salary
Workable
Senior/Lead Backend (NodeJS) Engineer - Onsite
About Deep Origin Led by Michael Antonov, a co-founder of Oculus, and well-funded by Formic Ventures, Deep Origin is poised to reinvent the way scientists work and life science innovations come to life. We see a future largely free of diseases, with a 150-year lifespan being a norm. To get there, we are building an operating system for science, enabling scientists to be more productive and to bring tomorrow's ideas to life quickly and at a reasonable cost. Applicants must be authorized to work for any employer in the U.S. We are unable to sponsor or take over sponsorship of an employment Visa at this time. Role Description In this hands-on position, you will be a key member of the software engineering team, building our key functionality and integrating with key partners. Your responsibilities will range from designing and developing complex, large-scale systems to writing APIs that integrate with various cloud providers and partners. You will have ownership in key software feature areas and their architectural design, as well as software implementation with a high level of independence and impact. Requirements 7+ years of experience designing, building, and operating complex, highly-scalable, distributed applications and systems 3+ years of hands-on software development experience with TypeScript/JavaScript/NodeJS Experience with both relational databases (e.g. Postgres) and NOSQL (e.g. MongoDB) Knowledge of Kubernetes and Cloud infrastructure/deployment tools (specifically with cluster operations and operators) Has built platforms from an early stage Has scaled platforms to handle many users (10,000+ DAU) Has extensive system-design experience Has experience designing systems with complex data-sets/relations Has experience working with distributed systems/platforms Thinks about architecture first and how the code fits in second Has experience working with/implementing a multi-tenant system Systematic problem-solving approach, coupled with a strong sense of ownership and drive Ability to work both independently and on the team Experience working in high-energy startups with fast product delivery mechanisms Benefits Benefits This position offers a competitive salary, benefits, and equity.
South San Francisco, CA, USA
Negotiable Salary
Workable
Software Dev Engineer IV - Python Developer
Position: Software Dev Engineer IV  Location: New York, NY, 10018  Duration: 6 Months       Job Type: Contract         Work Type:  Hybrid           Job Description:     Design, develop, implement, test, document and deploy full-stack, cloud-native, contact center-related software applications, tools, systems and services using multi-threaded programming, development in Python and React/node.js, implementing architecture patterns and design patterns, and utilizing generative AI large language models. Assist in gathering and analyzing business and functional requirements, and translate requirements into technical specifications for robust, scalable, supportable solutions that work well within the overall system architecture. Own delivery of entire piece of system or application, and serve as technical lead on complex projects using best practice engineering standards. Produce comprehensive, usable software documentation.  Qualifications:  MS or BS in Computer Science, Computer or Electrical Engineering, Mathematics, or a related field, plus five years of progressively responsible experience in the job offered or related occupations of Software Engineer, Software Developer, or related.  Required technical skills:  Coding proficiency in Python, and front-end development experience with Javascript/React.  Proficiency development with services such as AWS Lambda, Step Functions, DynamoDB, AppSync, Bedrock, SageMaker, and CloudWatch.  Proficiency in developing and integrating with REST-based or GraphQL-based APIs.  Proficiency in developing infrastructure-as-code deployment solutions such as AWS CloudFormation or AWS CDK .  Experience collaborating with other developers using git repositories, including creating and managing feature branches, pull requests, code merge, and GitHib actions or equivalent.  Preferred skills:  Experience with Contact Center development and telephony infrastructure.  Experience with prompt engineering for modern large language models.  Experience using modern AI-based agentic coding assistants for code development, test development, and documentation.  Track record of building successful serverless architectures following AWS Well Architected principles.  Candidate Requirements:  REQUIRED SKILLS:  Please refer required and technical skills in the job description that is what the manager needs on candidates resumes   Years of Experience: 5+   Degree or Certification:  Bachelors’ degree preferred  Top 3 must-have hard skills  Generative AI based coding  AWS serverless    Python and JavaScript/React 
New York, NY, USA
Negotiable Salary
Cookie
Cookie Settings
Our Apps
Download
Download on the
APP Store
Download
Get it on
Google Play
© 2025 Servanan International Pte. Ltd.