#NOTE: under construction. Many changes not here yet.

**classifieds-223**
- Category: human annotation error
- bugged eval functions: 'program_html'
- Issue: target page "url" for "program_html" is wrong: "index.php?page=item&id=12085" in the config.json file
- Fix: change "url" in config file to: "index.php?page=item&id=13215" 
- Explanation: wrong url leads to a page unrelated to the task. Evaluation script locators does not find the relevant title "Other cards" and Description "Are there any other Santa Claus themed envelopes?". Once corrected, oracles work.


**reddit-255**
- Category: (1) human annotation error; (2) intent mismatch
- Issues: 
    - (1) 'url_match' evaluator requires 'f/dataisbeautiful/124787' OR 'f/dataisbeautiful/124787/oc-nvidia-s-fy22-income-statement' in the URL. However, clicking on the image post leads to the zoom view of the image. This is also a valid destination, but the url for it does not include the strings above. The pattern is: "<...>/submission_images/e4345f0ff8286df3825a373faad0c73b389b107e1b1c90c05b5736e51699e96f.png'
    - (2) The intent says "Find me a post in /f/dataisbeautiful that illustrates the Nvidia revenue in FY22, and tell me the revenue of the whole company in FY22 (in billions).". If the agent does not navigate to the post, it can be marked as incorrect, even if the intent doesn't make that explicit.
- Fixes:
    - (1) Added the pattern |OR| `submission_images/e4345f0ff8286df3825a373faad0c73b389b107e1b1c90c05b5736e51699e96f.png` to the config file
    - (2) Added "To finish the task, please make sure to navigate to the page of the corresponding item." to the intent in the config file.
- Explanation: see issues.


**evaluation_harness/helper_functions.py, function `shopping_get_product_price`**
- Category:general code bug fix; helper function 
- Issue: Price parsing was brittle and failed for common locale formats (e.g., "1,234.56" and "1.234,56"), sometimes stripping separators incorrectly or losing decimals with multiple separators.
- Fix:
    - Rewrote the JS normalization logic:
        - Cleaning non-numeric characters in outertext
        - If both , and . exist, treat the rightmost separator as the decimal separator (e.g., "1.234,56" → "1234.56").
        - If only , exists, treat commas as thousands separators (remove all commas).
        - If only . exists, treat a single . as decimal; if multiple ., treat them as thousands separators (remove all).

- Explanation: see "issue" and fix. Examples now handled:
    - "1,234.56" -> 1234.56
    - "1.234,56" -> 1234.56
    - "1,234" -> 1234
    - "1.234.567.89" -> 123456789 (multiple . treated as thousands)

- NOTEs: 
    - When only commas, code defaults to thousands. Eg.: 4,999 -> 499; 5,99 -> 599. This is OK for shopping as it follows US format. Can include an heuristic to treat as thousand if <3 chars after ",".
    - Use `tests/test_evaluation_harness/test_number_parsing.py` for unit tests.

- Tasks affected: all config files using `func:shopping_get_product_price(__page__)`


**classifieds - "intent_template_id": 36 - "Navigate to my listing {{item}} and {{action}}."**
- Category: mismatch of intent and evaluator
- Issue: for many instances, 'action' doesn't include a instruction to submit the update, but evaluator requires it.
- Fix: added "\nPlease make sure to submit the changes." to intents when objective is not clear. 
- Tasks affected:
    - All classifieds tasks under the intent_template_id 36, except for classifieds_5: ""Navigate to my listing of the white car and delete it."
    - classifieds_4, classifieds_75, classifieds_76, classifieds_77, classifieds_159
- Example: 
    - Old: Navigate to my listing with a rug in the image and add to the description that there are no refunds. 
    - New: Navigate to my listing with a rug in the image and add to the description that there are no refunds.\nPlease make sure to submit the changes.",
- Explanation: Agents can just modify the contents and this would be a valid execution not covered by the evaluators. Can happen especially if the "update" button is not in the viewport, making evaluation not fair for smaller context window models.



# TODO: add remaining