arxiv-search-collector

---
name: arxiv-search-collector
description: "Model-driven arXiv retrieval workflow for building a paper set with a manual language parameter: initialize a run, fetch metadata for each model-designed query, let the model filter irrelevant items per query by keep indexes, then merge and dedupe into per-paper metadata directories. Use when query planning and relevance filtering should be done by the model, not rule-based heuristics."
---

# ArXiv Search Collector

Use this skill when you want model-led query planning and model-led relevance filtering.

## Core Principle

Scripts are tools. The model performs the reasoning and decisions:

1. Expand the original topic into multiple focused queries.
2. Run one fetch command per query.
3. Read each query result list and decide keep indexes.
4. Merge kept items and dedupe with one script.

## Step 1: Initialize Run

```bash
python3 scripts/init_collection_run.py \
  --output-root /path/to/data \
  --topic "LLM applications in Lean 4 formalization" \
  --keywords "Lean 4,LLM,formalization" \
  --categories "cs.AI,cs.LO" \
  --target-range 5-10 \
  --lookback 30d \
  --language English
```

This creates a run directory with `task_meta.json`, `task_meta.md`, `query_results/`, and `query_selection/`.

## Language Parameter

- `--language` must be set manually for each collection run.
- Use the same language value across all collector scripts for consistency.
- If `--language` is non-English (for example `Chinese`), generated markdown files are written in that language:
  - `task_meta.md`
  - `query_results/<label>.md`
  - `<arxiv_id>/metadata.md`
  - `papers_index.md`

## Query Writing Requirements

Follow these rules before running per-query fetch:

1. Determine query count from final target range.
- Prefer `3` queries for small/medium targets (`2-5`, `5-10`).
- Prefer `4` queries for larger targets (`10-50` or above).
- Avoid writing too many low-quality queries.

2. Alloc