Ta strona wyjaśnia, jak przesłać plik JSONL, uruchomić przetwarzanie wsadowe oraz pobrać wyniki za pośrednictwem API CLARIN-PL. Przetwarzanie wsadowe umożliwia wykonywanie w tle wielu niezależnych żądań do modelu. Przesyłasz plik, od czasu do czasu sprawdzasz status przetwarzania, a po jego zakończeniu pobierasz wyniki. Rozwiązanie jest przeznaczone do dużych, nieinteraktywnych zadań, takich jak klasyfikacja, ocenianie lub tłumaczenie dużej liczby elementów.
CLARIN-PL Services / Wsady: https://services.clarin-pl.eu/app/batches.
Wszystkie żądania są kierowane do https://services.clarin-pl.eu/api/v1 z nagłówkiem Authorization: Bearer <API token>. Skopiuj token ze strony API access.
Możesz również przesłać plik w sekcji Storage i uruchomić przetwarzanie wsadowe bezpośrednio ze strony internetowej.
Plik wejściowy to plik tekstowy w kodowaniu UTF-8 z rozszerzeniem .jsonl zawierający jedno żądanie JSON w każdym wierszu. Każdy wiersz jest przetwarzany jako osobne żądanie.
custom_id musi być unikalny. Służy on do powiązania wyniku z odpowiednim żądaniem.POST do /chat/completions lub /responses (url może zawierać prefiks /v1, ale nie musi).stream lub ustaw jego wartość na false.Format żądania Chat Completions. Użyj go, jeśli Twój kod już korzysta z /v1/chat/completions. Każde żądanie opakuj w custom_id.
{"custom_id":"request-1","method":"POST","url":"/v1/chat/completions","body":{"model":"model-name","messages":[{"role":"user","content":"Hello"}]}}
{"custom_id":"request-2","method":"POST","url":"/v1/chat/completions","body":{"model":"model-name","messages":[{"role":"user","content":"Summarise this text."}]}}
Format żądania Responses API. Ten sam format opakowania, ale dla nowszego interfejsu Responses API (/v1/responses). Przetwarzanie wsadowe sprawdza jedynie, czy pola body.model (niepusty ciąg znaków) i body.input zostały podane.
{"custom_id":"req-3","method":"POST","url":"/v1/responses","body":{"model":"model-name","input":"Hello"}}
Bezpośrednia treść żądania Chat Completions. Najprostsza opcja: samo ciało żądania, bez dodatkowego opakowania. Jest ono zawsze wysyłane do /chat/completions, a wartość custom_id każdego wyniku odpowiada numerowi wiersza liczonemu od 0 ("0", "1", "2", …).
{"model":"model-name","messages":[{"role":"user","content":"Hello"}]}
| Pole | Wymagane | Dozwolone wartości |
|---|---|---|
model |
tak | niepusty ciąg znaków |
messages |
tak | niepusta lista |
messages[].role |
tak | system, user, assistant, tool |
messages[].content |
tak (z wyjątkiem assistant) |
ciąg znaków, lista elementów text / image_url lub null |
temperature |
nie | liczba 0–2 |
top_p |
nie | liczba 0–1 |
max_tokens |
nie | liczba całkowita > 0 |
logprobs |
nie | wartość logiczna |
top_logprobs |
nie | liczba całkowita ≥ 0 |
tools |
nie | lista |
response_format, metadata |
nie | obiekt |
curl -X POST \
-H "Authorization: Bearer <API token>" \
-F "f=@input.jsonl;type=application/jsonl" \
"https://services.clarin-pl.eu/api/v1/files/batches/input.jsonl"
Response: {"batches/input.jsonl": "Uploaded"}.
The part of the URL after /files/ is the file path in your storage. Here it is batches/input.jsonl, and that is the value you use as input_file_path later. Upload batch input only through /files. Files uploaded through /oapi/files cannot be used in a batch.
Uploads never overwrite files. If the path is already taken, you get 409. Choose a different name or delete the old file first:
curl -X DELETE -H "Authorization: Bearer <API token>" \
"https://services.clarin-pl.eu/api/v1/files/batches/input.jsonl"
curl -X POST \
-H "Authorization: Bearer <API token>" \
-F "input_file_path=batches/input.jsonl" \
"https://services.clarin-pl.eu/api/v1/oapi/batches/verify_file"
Instead of input_file_path you can send the local file with -F "file=@input.jsonl", but not both at once.
Example response for an invalid file:
{
"valid": false,
"total": 2,
"max_items": 40000,
"errors": [
{"line": 2, "message": "body.messages[0].role must be one of system, user, assistant, tool"}
]
}
line is counted from 1 and skips blank lines. It is null for problems with the whole file (for example wrong encoding or extension).
curl -X POST \
-H "Authorization: Bearer <API token>" \
-H "Content-Type: application/json" \
-d '{"input_file_path":"batches/input.jsonl"}' \
"https://services.clarin-pl.eu/api/v1/oapi/batches"
Optionally, add "output_file_path":"batches/my_output.jsonl" to choose where the results are saved. The path must end with .jsonl and must not exist yet. Without it, results go to <input name>_output_<batch id>.jsonl next to the input file. Errors always go to <input name>_error_<batch id>.jsonl.
The call returns immediately:
{
"id": "3f2b9c1e-8a4d-4c7e-9b1a-2d5e6f7a8b9c",
"status": "validating",
"input_file_path": "batches/input.jsonl",
"output_file_path": "batches/input_output_3f2b9c1e-8a4d-4c7e-9b1a-2d5e6f7a8b9c.jsonl",
"error_file_path": "batches/input_error_3f2b9c1e-8a4d-4c7e-9b1a-2d5e6f7a8b9c.jsonl",
"counts": {"total": 0, "pending": 0, "running": 0, "succeeded": 0, "failed": 0, "expired": 0},
"error": null,
...
}
Save id, output_file_path, and error_file_path. The file content is checked after this call, so an invalid file does not return an error here. Instead, the batch changes to failed and the reason appears in error. Run section 3.2 first to see problems with line numbers.
curl -H "Authorization: Bearer <API token>" \
"https://services.clarin-pl.eu/api/v1/oapi/batches/<batch id>"
Check it whenever you need and look at:
status: see section 4.1.counts: how many lines are pending, running, succeeded, failed, or expired.error: why the batch failed.estimated_completion_at: expected finish time (Unix timestamp), available while in_progress.To list all your batches, call the same URL without /<batch id>.
Use the paths from the batch response:
curl -H "Authorization: Bearer <API token>" \
"https://services.clarin-pl.eu/api/v1/files/<output_file_path>?mode=download" -o output.jsonl
curl -H "Authorization: Bearer <API token>" \
"https://services.clarin-pl.eu/api/v1/files/<error_file_path>?mode=download" -o errors.jsonl
You do not have to wait for the end. Every time another 10% of the lines is finished, both files are updated, so you can download partial results while the batch is running. A 404 means the file does not exist yet: no 10% step has been reached, or (for the error file) no line has failed.
curl -X POST -H "Authorization: Bearer <API token>" \
"https://services.clarin-pl.eu/api/v1/oapi/batches/<batch id>/pause"
curl -X POST -H "Authorization: Bearer <API token>" \
"https://services.clarin-pl.eu/api/v1/oapi/batches/<batch id>/resume"
curl -X POST -H "Authorization: Bearer <API token>" \
-H "Content-Type: application/json" -d '{"message":"no longer needed"}' \
"https://services.clarin-pl.eu/api/v1/oapi/batches/<batch id>/cancel"
Cancelling cannot be undone, and it does not save the final files. Only the results from the last 10% update stay in your storage. If you want to keep everything processed so far, download the files first or use pause instead. The message is optional.
If an action does not fit the current status (for example cancelling a finished batch), the batch is returned unchanged with 200.
| Status | Meaning | Final |
|---|---|---|
validating |
the file is being checked, right after the batch is created | no |
in_progress |
lines are being processed | no |
paused |
paused, can be resumed | no |
completed |
finished and at least one line succeeded. Some lines may still have failed, so check counts and the error file |
yes |
failed |
the file was rejected (reason in error) or no line succeeded |
yes |
expired |
the 24-hour limit passed. Unfinished lines are in the error file | yes |
cancelled |
cancelled (reason in status_message) |
yes |
Results are saved in the order they finish, not in the input order. Match them by custom_id.
Output file, one line per successful request (body is the full model response):
{"custom_id":"req-1","response":{"status_code":200,"body":{...}},"error":null}
Error file, one line per failed request (created only if something fails):
{"custom_id":"req-2","response":null,"error":{"status_code":500,"message":"..."}}
error.status_code |
Meaning |
|---|---|
408 |
the batch expired before this line was processed |
504 |
the line timed out after all retries |
500 |
other processing error |
other 4xx |
the model service rejected the request (for example invalid request or exceeded quota) |
Temporary failures are retried automatically before a line is written to the error file.
All endpoints used in this guide, relative to https://services.clarin-pl.eu/api/v1. Each request needs the Authorization: Bearer <API token> header.
POST /files/{file_path} – uploads a new file (does not overwrite)GET /files/{file_path}?mode=download – downloads a fileDELETE /files/{file_path} – deletes a filePOST /oapi/batches – creates a batch from an input JSONL file pathGET /oapi/batches – lists the current user's batchesPOST /oapi/batches/verify_file – validates a JSONL file (from input_file_path or multipart file/f)GET /oapi/batches/{batch_id} – returns one batchPOST /oapi/batches/{batch_id}/cancel – cancels a batch (optional {"message": "reason"})POST /oapi/batches/{batch_id}/pause – pauses a validating or in_progress batchPOST /oapi/batches/{batch_id}/resume – resumes a paused batch| Code | Cause | What to do |
|---|---|---|
400 |
create: input_file_path does not end with .jsonl, or output_file_path is invalid or already exists; validate: both or neither of input_file_path and file were sent |
read detail and fix the request |
401 |
missing or invalid token | copy the token again from API access |
403 |
storage limit exceeded | delete files you no longer need |
404 |
the batch or file does not exist, or belongs to another user | check the id or path |
409 |
upload: a file with this path already exists | use another name or delete the old file |
422 |
a required field is missing (for example f in upload or input_file_path in create) or the body is not valid JSON |
fix the request |
429 |
more than 240 requests per minute, or already 10 active batches | wait and try again |