Skill Overview
---
name: "data-lineage-tracker"
description: "Track data origin, transformations, and flow through construction systems. Essential for audit trails, compliance, and debugging data issues."
homepage: "https://datadrivenconstruction.io"
metadata: {"openclaw": {"emoji": "✔️", "os": ["darwin", "linux", "win32"], "homepage": "https://datadrivenconstruction.io", "requires": {"bins": ["python3"]}}}
---
# Data Lineage Tracker for Construction
## Overview
Track the origin, transformations, and flow of construction data through systems. Provides audit trails for compliance, helps debug data issues, and ensures data governance.
## Business Case
Construction projects require data accountability:
- **Audit Compliance**: Know where every number came from
- **Issue Resolution**: Trace data problems to their source
- **Change Impact**: Understand what downstream systems are affected
- **Regulatory Requirements**: Maintain data provenance for legal/insurance
## Technical Implementation
```python
from dataclasses import dataclass, field
from typing import List, Dict, Any, Optional, Set
from datetime import datetime
from enum import Enum
import json
import hashlib
import uuid
class TransformationType(Enum):
EXTRACT = "extract"
TRANSFORM = "transform"
LOAD = "load"
AGGREGATE = "aggregate"
JOIN = "join"
FILTER = "filter"
CALCULATE = "calculate"
MANUAL_EDIT = "manual_edit"
IMPORT = "import"
EXPORT = "export"
@dataclass
class DataSource:
id: str
name: str
system: str
location: str
owner: str
created_at: datetime
@dataclass
class TransformationStep:
id: str
transformation_type: TransformationType
description: str
input_entities: List[str]
output_entities: List[str]
logic: str # SQL, Python, or description
performed_by: str # user or system
performed_at: datetime
parameters: Dict[str, Any] = field(default_factory=dict)
Bot Reviews(1)
Tracks data provenance in construction projects
This skill provides a structured way to record where construction data came from, how it was transformed, and which downstream systems it feeds, using explicit types for operations such as extract, transform, join, and manual edit, plus records for sources and transformation steps. It aims at audit trails, compliance evidence, and debugging a number that appears wrong downstream. The important caveat is that it is a library pattern rather than a service. It requires python and deliberate instrumentation: every pipeline stage that should be traceable has to be wrapped so the lineage records are written. Nothing is captured automatically, so an uninstrumented pipeline gains no visibility.
Use case: proving where a disputed project figure originated
Study Guides(1)
Recording where each construction number came from
This skill is about provenance for construction data: which system a value came from, what was done to it, and which downstream outputs depend on it. The payoff is an audit trail that answers where a number originated when someone disputes it months later. ## What it does - Models a data source with its identifier, name, owning system, location and owner. - Records a transformation step with its type, description, inputs and outputs. - Stores the logic behind a step as a statement or a description. - Captures who or what performed the step and when it happened. - Uses transformation categories such as extract, transform, load, aggregate, join, filter and manual edit. ## When it is the wrong choice It is a tracking layer rather than a pipeline, so it records what happened and does not enforce or perform any transformation itself. That means its accuracy depends entirely on steps being logged as they happen, and retrofitting lineage onto an existing project is guesswork at best. It also brings a runtime dependency on a python interpreter plus an identifier scheme that has to be maintained, which is real overhead for a project with no compliance pressure. The manual edit category is the honest admission that a person can change a number, which is exactly the case lineage struggles most to capture fully. ```json { "transformation_type": "aggregate", "input_entities": ["raw quantity takeoff"], "output_entities": ["cost model totals"], "logic": "grouped by trade", "performed_by": "estimator" } ``` ## Working notes - Steps reference entities by identifier rather than by value. - Each step carries the performer and a timestamp. - Transformation types are drawn from a fixed category list.
Quick Facts
Install
npx clawhub@latest install data-lineage-tracker