Skip to content

Commit cc20711

Browse files
hkad98hkad98
committed
fix(docs): fix docs build for old branches and broken links
Several issues with the V2 docs pipeline building old release branches: 1. Always use griffe_builder.py and method_page_renderer.py from the current branch instead of checking if they exist on the target branch. The scripts were always executed from the current branch anyway, and griffe does static analysis via --search-path (no imports needed). This removes the json_builder.py fallback that caused ModuleNotFoundError and the pip install -e workaround. 2. Restore Hugo shortcode templates (parameters-block, parameter) that were deleted in the griffe pipeline commit. Old release branches have hand-written method pages using these shortcodes without api_ref frontmatter, so method_page_renderer skips them and Hugo needs the templates to render them. 3. Fix python_ref_builder generating only one page for modules/classes sharing a leaf name (e.g. declarative_model under data_source, workspace, user, permission). The "if name not in links" guard prevented page creation for all but the first. Now all modules and classes always get their pages while the links dict keeps first-wins for type-annotation resolution. 4. Use relative links (href="child_name/") for module child listings and class method/property tables instead of the global links dict. This fixes cross-links where e.g. load_from_disk in CatalogDeclarativeDataSources pointed to workspace's version. 5. Temporarily disable version caching while stabilizing the pipeline. JIRA: trivial risk: nonprod
1 parent e650ab3 commit cc20711

6 files changed

Lines changed: 93 additions & 73 deletions

File tree

‎.github/workflows/netlify-deploy-v2.yaml‎

Lines changed: 13 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -39,39 +39,37 @@ jobs:
3939
SHA=$(gh api "repos/${{ github.repository }}/git/ref/heads/${{ matrix.version.branch }}" -q '.object.sha')
4040
echo "sha=$SHA" >> $GITHUB_OUTPUT
4141
echo "Branch ${{ matrix.version.branch }} -> section ${{ matrix.version.section }} (SHA: $SHA)"
42-
- name: Cache version docs
43-
id: cache
44-
uses: actions/cache@v4
45-
with:
46-
path: docs/versioned_docs/${{ matrix.version.section }}
47-
key: version-docs-${{ hashFiles('scripts/docs/*.py', 'scripts/docs/templates/**', 'docs/*_template.md') }}-${{ matrix.version.section }}-${{ steps.sha.outputs.sha }}
42+
# TODO: re-enable once the pipeline is stable
43+
# - name: Cache version docs
44+
# id: cache
45+
# uses: actions/cache@v4
46+
# with:
47+
# path: docs/versioned_docs/${{ matrix.version.section }}
48+
# key: version-docs-${{ hashFiles('scripts/docs/*.py', 'scripts/docs/templates/**', 'docs/*_template.md') }}-${{ matrix.version.section }}-${{ steps.sha.outputs.sha }}
4849
- name: Checkout
49-
if: steps.cache.outputs.cache-hit != 'true'
50+
5051
uses: actions/checkout@v4
5152
- name: Fetch target branch
52-
if: steps.cache.outputs.cache-hit != 'true'
53+
5354
run: git fetch origin ${{ matrix.version.branch }}
5455
- name: Checkout branch packages
55-
if: steps.cache.outputs.cache-hit != 'true'
56+
5657
run: |
5758
git checkout origin/${{ matrix.version.branch }} -- gooddata-api-client/ packages/gooddata-sdk/ packages/gooddata-pandas/
5859
- name: Setup Python
59-
if: steps.cache.outputs.cache-hit != 'true'
60+
6061
uses: actions/setup-python@v5
6162
with:
6263
python-version-file: ".python-version"
6364
cache: 'pip'
6465
cache-dependency-path: scripts/script-requirements.txt
6566
- name: Install Dependencies
66-
if: steps.cache.outputs.cache-hit != 'true'
67+
6768
run: |
6869
python -m pip install --upgrade pip
6970
pip install -r scripts/script-requirements.txt
70-
# json_builder.py (legacy fallback for branches without griffe_builder.py)
71-
# imports gooddata_sdk and gooddata_pandas at runtime, so they must be installed.
72-
pip install -e gooddata-api-client/ -e packages/gooddata-sdk/ -e packages/gooddata-pandas/
7371
- name: Generate version docs
74-
if: steps.cache.outputs.cache-hit != 'true'
72+
7573
run: bash scripts/generate-single-version.sh "origin/${{ matrix.version.branch }}" "${{ matrix.version.section }}"
7674
- name: Upload version artifact
7775
uses: actions/upload-artifact@v4

‎docs/layouts/shortcodes/parameter.html‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
<tr>
2+
{{ with .Get "p_name" }}<th padding="0px">{{ . | safeHTML }}{{ end }}
3+
{{ with .Get "p_type" }}<th padding="0px">{{ . | safeHTML }}{{ end }}
4+
<th>{{.Inner | safeHTML}}
5+
</tr>

‎docs/layouts/shortcodes/parameters-block.html‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
{{ $partitle := .Get "title"}}
2+
<h4>{{$partitle}}</h4>
3+
{{ with .Get `None`}}
4+
<i>None</i>
5+
{{else}}
6+
<table class="gd-docs-parameters-block">
7+
<thead>
8+
<tr>
9+
{{ if eq $partitle "Parameters"}}
10+
<th>name</th>
11+
{{end}}
12+
<th>type</th>
13+
<th>description</th>
14+
</tr>
15+
</thead>
16+
<tbody>
17+
{{.Inner}}
18+
</tbody>
19+
</table>
20+
{{end}}

‎scripts/docs/python_ref_builder.py‎

Lines changed: 27 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -242,12 +242,14 @@ def render_class_html(class_data: dict, parent_name: str, import_path: str, reso
242242
continue
243243
fds = fdata.get("docstring_parsed")
244244
desc = resolver.all_links(fds.get("short_description", "")) if fds else ""
245+
# Link to the method's own child page (relative), not the global links dict
246+
local_link = f'<a href="{fname.lower()}/">{fname}</a>'
245247
if fdata.get("is_property"):
246-
properties.append({"name_link": resolver.type_link(fname), "description": desc})
248+
properties.append({"name_link": local_link, "description": desc})
247249
else:
248250
methods.append(
249251
{
250-
"name_link": resolver.type_link(fname),
252+
"name_link": local_link,
251253
"signature": _function_signature(fdata),
252254
"description": desc,
253255
}
@@ -263,7 +265,9 @@ def render_module_html(module_data: dict, resolver: LinkResolver) -> str:
263265
for obj_name, obj_data in module_data.items():
264266
if obj_name == "kind" or not isinstance(obj_data, dict):
265267
continue
266-
entries.append({"kind": obj_data.get("kind", ""), "name_link": resolver.type_link(obj_name)})
268+
# Link to the child page (relative), not the global links dict
269+
local_link = f'<a href="{obj_name.lower()}/">{obj_name}</a>'
270+
entries.append({"kind": obj_data.get("kind", ""), "name_link": local_link})
267271
return _MODULE_TPL.render(entries=entries)
268272

269273

@@ -333,34 +337,34 @@ def _pass1(data_root: dict, dir_root: Path, api_ref_root: str, module_import_pat
333337
obj_module_import_path = obj_module_import_path.replace(".functions", "")
334338

335339
if kind == "module":
340+
(dir_root / name).mkdir(exist_ok=True)
336341
if name not in links:
337-
(dir_root / name).mkdir(exist_ok=True)
338342
links[name] = {"path": f"{api_ref_root}/{name}".lower(), "kind": "function"}
339-
pages.append(
340-
_PageSpec(
341-
kind="module",
342-
name=name,
343-
parent_name="",
344-
import_path=obj_module_import_path,
345-
file_path=dir_root / name / "_index.md",
346-
data=obj,
347-
)
343+
pages.append(
344+
_PageSpec(
345+
kind="module",
346+
name=name,
347+
parent_name="",
348+
import_path=obj_module_import_path,
349+
file_path=dir_root / name / "_index.md",
350+
data=obj,
348351
)
352+
)
349353

350354
elif kind == "class":
355+
(dir_root / name).mkdir(exist_ok=True)
351356
if name not in links:
352-
(dir_root / name).mkdir(exist_ok=True)
353357
links[name] = {"path": f"{api_ref_root}/{name}".lower(), "kind": "class"}
354-
pages.append(
355-
_PageSpec(
356-
kind="class",
357-
name=name,
358-
parent_name=module_import_path.split(".")[-1],
359-
import_path=obj_module_import_path,
360-
file_path=dir_root / name / "_index.md",
361-
data=obj,
362-
)
358+
pages.append(
359+
_PageSpec(
360+
kind="class",
361+
name=name,
362+
parent_name=module_import_path.split(".")[-1],
363+
import_path=obj_module_import_path,
364+
file_path=dir_root / name / "_index.md",
365+
data=obj,
363366
)
367+
)
364368

365369
elif name == "functions":
366370
for func_name in obj:

‎scripts/generate-single-version.sh‎

Lines changed: 15 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -54,31 +54,27 @@ if git cat-file -e "$GRIFFE_GEN_FILE" 2>/dev/null || git cat-file -e "$LEGACY_GE
5454
rm -f api_spec.toml
5555
fi
5656

57-
# Generate API introspection data — prefer griffe (static analysis, no imports needed)
58-
if git cat-file -e "$GRIFFE_GEN_FILE" 2>/dev/null; then
59-
echo "Using griffe_builder.py (static analysis)"
60-
python3 ../scripts/docs/griffe_builder.py \
61-
--search-path ../packages/gooddata-sdk/src \
62-
--search-path ../packages/gooddata-pandas/src \
63-
--output data.json \
64-
gooddata_sdk gooddata_pandas
65-
else
66-
echo "Falling back to json_builder.py (runtime introspection)"
67-
python3 ../scripts/docs/json_builder.py
68-
fi
57+
# Generate API introspection data using griffe (static analysis, no imports needed).
58+
# Always use the current branch's griffe_builder.py — it works on any branch's
59+
# source code via --search-path and doesn't require the SDK packages to be installed.
60+
python3 ../scripts/docs/griffe_builder.py \
61+
--search-path ../packages/gooddata-sdk/src \
62+
--search-path ../packages/gooddata-pandas/src \
63+
--output data.json \
64+
gooddata_sdk gooddata_pandas
6965

7066
# Generate API reference markdown files and export links for method page renderer
7167
python3 ../scripts/docs/python_ref_builder.py api_spec.toml \
7268
data.json "$section" "$content_dir" \
7369
--export-links links.json
7470

75-
# Pre-render method pages with api_ref directives
76-
if git cat-file -e "$branch:scripts/docs/method_page_renderer.py" 2>/dev/null; then
77-
echo "Pre-rendering method pages for section $section..."
78-
python3 ../scripts/docs/method_page_renderer.py \
79-
data.json "$content_dir/$section" \
80-
--links-json links.json
81-
fi
71+
# Pre-render method pages with api_ref directives.
72+
# Always use the current branch's renderer — old branches have Hugo shortcodes
73+
# (parameters-block, parameter) whose templates were removed.
74+
echo "Pre-rendering method pages for section $section..."
75+
python3 ../scripts/docs/method_page_renderer.py \
76+
data.json "$content_dir/$section" \
77+
--links-json links.json
8278

8379
# Clean up intermediate files (no longer needed after pre-rendering)
8480
rm -f data.json links.json

‎scripts/generate.sh‎

Lines changed: 13 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -103,24 +103,21 @@ for branch in "${branches_to_process[@]}" ; do
103103
echo "removing the API_spec"
104104
rm -rf api_spec.toml
105105
fi
106-
# Prefer griffe (static analysis, no imports needed)
107-
if git cat-file -e "$GRIFFE_GEN_FILE" 2>/dev/null; then
108-
python3 ../scripts/docs/griffe_builder.py \
109-
--search-path ../packages/gooddata-sdk/src \
110-
--search-path ../packages/gooddata-pandas/src \
111-
--output data.json \
112-
gooddata_sdk gooddata_pandas
113-
else
114-
python3 ../scripts/docs/json_builder.py
115-
fi
106+
# Always use griffe (static analysis, no imports needed).
107+
# Works on any branch's source code via --search-path.
108+
python3 ../scripts/docs/griffe_builder.py \
109+
--search-path ../packages/gooddata-sdk/src \
110+
--search-path ../packages/gooddata-pandas/src \
111+
--output data.json \
112+
gooddata_sdk gooddata_pandas
116113
python3 ../scripts/docs/python_ref_builder.py api_spec.toml data.json "$target_section" versioned_docs \
117114
--export-links links.json
118-
# Pre-render method pages with api_ref directives
119-
if git cat-file -e "$branch:scripts/docs/method_page_renderer.py" 2>/dev/null; then
120-
python3 ../scripts/docs/method_page_renderer.py \
121-
data.json "versioned_docs/$target_section" \
122-
--links-json links.json
123-
fi
115+
# Pre-render method pages with api_ref directives.
116+
# Always use the current branch's renderer — old branches have Hugo shortcodes
117+
# (parameters-block, parameter) whose templates were removed.
118+
python3 ../scripts/docs/method_page_renderer.py \
119+
data.json "versioned_docs/$target_section" \
120+
--links-json links.json
124121
rm -f data.json links.json
125122
fi
126123
fi

0 commit comments

Comments
 (0)