Development Workflow

This guide covers building from source, running tests, and contributing changes to OpenDataLoader PDF.

Prerequisites

Before you begin, ensure you have the following installed:

Verify your setup:

java -version
mvn --version
python --version
uv --version
node --version
pnpm --version

OS-Specific Install Commands

Windows users: We recommend WSL 2 for the smoothest development experience. All shell scripts (./scripts/*.sh) assume a Unix-like environment.

Git LFS

Some test fixtures are stored with Git LFS. Install it before cloning:

# macOS
brew install git-lfs

# Ubuntu/Debian
sudo apt install git-lfs

# Then initialize
git lfs install

Build & Test

Quick Start (Local Development)

Run tests for each package independently:

# Java tests
./scripts/test-java.sh

# Python tests
./scripts/test-python.sh

# Node.js tests
./scripts/test-node.sh

Full CI Build

Build all packages (Java, Python, Node.js) in one command:

./scripts/build-all.sh

Build Java Only

mvn clean install -f java/pom.xml

Successful builds produce artifacts under java/opendataloader-pdf-cli/target, including the shaded CLI JAR.

Run the CLI from Source

After building, run the CLI directly:

java -jar java/opendataloader-pdf-cli/target/opendataloader-pdf-cli-<VERSION>.jar [options] <INPUT>

Refer to the CLI Options Reference for the full flag list.

Code Generation

Warning: After changing CLI options in Java, you must run npm run sync. This regenerates options.json and all Python/Node.js bindings. Forgetting this silently breaks the wrappers.

CLI options and JSON schema documentation are auto-generated from source files. This ensures consistency across all language bindings.

Note: Reference documentation MDX files (CLI options, JSON schema, convert options) are generated by CI at release time and pushed to the opendataloader.org repository. They are not tracked in this repo. Manual documentation also lives in opendataloader.org.

Auto-Generated Files (Do Not Edit)

The following files are generated by npm run sync — edit the Java source instead:

• options.json
• node/opendataloader-pdf/src/cli-options.generated.ts
• node/opendataloader-pdf/src/convert-options.generated.ts
• python/opendataloader-pdf/src/opendataloader_pdf/cli_options_generated.py
• python/opendataloader-pdf/src/opendataloader_pdf/convert_generated.py

Available Commands

After Modifying Java CLI Options

npm run sync-options

This exports options from Java and generates:

After Modifying JSON Schema

Edit schema.json directly, then:

npm run generate-schema

This generates:

Full Sync

To regenerate everything (options + schema):

npm run sync

Project Structure

opendataloader-pdf/
├── java/                          # Core Java engine
│   ├── opendataloader-pdf-core/   # Main library
│   └── opendataloader-pdf-cli/    # CLI application
├── python/                        # Python package
├── node/                          # Node.js package
└── scripts/                       # Build & test scripts

Code Style

• Java: Follow existing patterns in the codebase
• Python: PEP 8 with type hints
• TypeScript: ESLint configuration in project

Resources

• CLI Options Reference — All available command-line options
• JSON Schema — Output format specification
• Javadoc — Java API reference
• Contributing Guide — How to submit changes