API throttled when deploying large number of notebooks #704
Description
Library Version
0.1.33
What is the problem?
During the deployment of over 200 Notebooks we see heavy API throttling which leads to a runtime for the deployment over 1h which kills my Azure DevOps deployment pipeline, unfortunately.
I have run the Fabric CI/CD with DEBUG enabled, here is an excerpt.
2026-01-02T13:05:27.4903017Z 13:05:27 - API is throttled. Checking again in 20 seconds (Attempt 1)...�[0m
2026-01-02T13:05:47.4906258Z �[30m[debug] 13:05:47 -
2026-01-02T13:05:47.4909358Z URL: https://api.powerbi.com/v1/workspaces/49b73202-2d19-442e-94a5-e89f868a44b2/lakehouses/bb14b91b-975f-45fe-b4f8-d1b8756643ed
2026-01-02T13:05:47.4909922Z Method: GET
2026-01-02T13:05:47.4910287Z Request Body:
2026-01-02T13:05:47.4910626Z "{}"
2026-01-02T13:05:47.4910989Z Response Status: 429
2026-01-02T13:05:47.4911379Z Response Headers:
2026-01-02T13:05:47.4911767Z {
2026-01-02T13:05:47.4912081Z "Content-Length": "169",
2026-01-02T13:05:47.4912448Z "Content-Type": "application/json; charset=utf-8",
2026-01-02T13:05:47.4912782Z "Retry-After": "53",
2026-01-02T13:05:47.4913099Z "x-ms-public-api-error-code": "RequestBlocked",
2026-01-02T13:05:47.4913474Z "request-redirected": "true",
2026-01-02T13:05:47.4913870Z "home-cluster-uri": "https://wabi-north-europe-p-primary-redirect.analysis.windows.net/",
2026-01-02T13:05:47.4914365Z "RequestId": "b3c96052-d345-4d15-bb6e-07fb6b953f7a",
2026-01-02T13:05:47.4914818Z "Date": "Fri, 02 Jan 2026 13:05:26 GMT"
2026-01-02T13:05:47.4915184Z }
2026-01-02T13:05:47.4915443Z Response Body:
2026-01-02T13:05:47.4915864Z {"requestId":"b3c96052-d345-4d15-bb6e-07fb6b953f7a","errorCode":"RequestBlocked","message":"Request is blocked by the upstream service until: 1/2/2026 1:06:20 PM (UTC)"}
2026-01-02T13:05:47.4916312Z �[0m
2026-01-02T13:05:47.5905072Z 13:05:47 - API is throttled. Checking again in 33 seconds (Attempt 2)...�[0m
2026-01-02T13:06:20.5907708Z �[30m[debug] 13:06:20 -
2026-01-02T13:06:20.5909271Z URL: https://api.powerbi.com/v1/workspaces/49b73202-2d19-442e-94a5-e89f868a44b2/lakehouses/bb14b91b-975f-45fe-b4f8-d1b8756643ed
2026-01-02T13:06:20.5910088Z Method: GET
2026-01-02T13:06:20.5910458Z Request Body:
2026-01-02T13:06:20.5910783Z "{}"
2026-01-02T13:06:20.5911147Z Response Status: 429
2026-01-02T13:06:20.5911537Z Response Headers:
2026-01-02T13:06:20.5911890Z {
2026-01-02T13:06:20.5912246Z "Content-Length": "169",
2026-01-02T13:06:20.5912613Z "Content-Type": "application/json; charset=utf-8",
2026-01-02T13:06:20.5912951Z "Retry-After": "33",
2026-01-02T13:06:20.5913277Z "x-ms-public-api-error-code": "RequestBlocked",
2026-01-02T13:06:20.5913630Z "request-redirected": "true",
2026-01-02T13:06:20.5914010Z "home-cluster-uri": "https://wabi-north-europe-p-primary-redirect.analysis.windows.net/",
2026-01-02T13:06:20.5914471Z "RequestId": "b5bc3086-52cc-412e-9f37-d29e91cad96f",
2026-01-02T13:06:20.5914960Z "Date": "Fri, 02 Jan 2026 13:05:47 GMT"
2026-01-02T13:06:20.5915336Z }
2026-01-02T13:06:20.5915652Z Response Body:
2026-01-02T13:06:20.5916091Z {"requestId":"b5bc3086-52cc-412e-9f37-d29e91cad96f","errorCode":"RequestBlocked","message":"Request is blocked by the upstream service until: 1/2/2026 1:06:20 PM (UTC)"}
[... happens a few more times ...]
2026-01-02T13:06:22.8970452Z �[30m[debug] 13:06:22 - Found attribute 'id' with value 'fe1ac118-9bff-4174-a320-95601009d2ef'�[0m
2026-01-02T13:06:22.8973134Z �[30m[debug] 13:06:22 - Replacing '21ac452f-0644-40da-bfca-526224ff66c5' with 'fe1ac118-9bff-4174-a320-95601009d2ef' in NB_Bronze_D365FO_Synlink.Notebook�[0m
2026-01-02T13:06:22.8973810Z �[30m[debug] 13:06:22 - No optional filters found. Find and replace applied in this repository file�[0m
2026-01-02T13:06:22.8974398Z �[30m[debug] 13:06:22 - No optional filters found. Find and replace applied in this repository file�[0m
2026-01-02T13:06:22.8976684Z �[30m[debug] 13:06:22 - No optional filters found. Find and replace applied in this repository file�[0m
2026-01-02T13:06:22.8979033Z �[30m[debug] 13:06:22 - Replacing '0ed03902-482a-47cf-9a06-342bb1ce531a' with '49b73202-2d19-442e-94a5-e89f868a44b2' in NB_Bronze_D365FO_Synlink.Notebook�[0m
2026-01-02T13:06:22.8979607Z �[30m[debug] 13:06:22 - Item type match found: True�[0m
2026-01-02T13:06:22.8980175Z �[30m[debug] 13:06:22 - Processing $items variable with item_type=Lakehouse, item_name=LH_Bronze_Platform, attribute=id�[0m
All notebooks may or may not reference a default lakehouse that is located in a different workspace. This is the relevant part of my parameter.yml:
# Lakehouse-ID
- find_value: \#\s*META\s+"default_lakehouse":\s*"([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})",\s*\#\s*META\s+"default_lakehouse_name":\s*"LH_Bronze_Platform"
replace_value:
test: "$items.Lakehouse.LH_Bronze_Platform.$id"
prod: "$items.Lakehouse.LH_Bronze_Platform.$id"
# Optional field: Set to "true" to treat find_value as a regex pattern
is_regex: "true" # "<true|True>"
item_type: "Notebook" # filter on notebook files
- find_value: \#\s*META\s+"default_lakehouse":\s*"([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})",\s*\#\s*META\s+"default_lakehouse_name":\s*"LH_Silver_Platform"
replace_value:
test: "$items.Lakehouse.LH_Silver_Platform.$id"
prod: "$items.Lakehouse.LH_Silver_Platform.$id"
# Optional field: Set to "true" to treat find_value as a regex pattern
is_regex: "true" # "<true|True>"
item_type: "Notebook" # filter on notebook files
- find_value: \#\s*META\s+"default_lakehouse":\s*"([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})",\s*\#\s*META\s+"default_lakehouse_name":\s*"LH_Gold_Platform"
replace_value:
test: "$items.Lakehouse.LH_Gold_Platform.$id"
prod: "$items.Lakehouse.LH_Gold_Platform.$id"
# Optional field: Set to "true" to treat find_value as a regex pattern
is_regex: "true" # "<true|True>"
item_type: "Notebook" # filter on notebook files
# Lakehouse-Workspace-ID
- find_value: \#\s*META\s+"default_lakehouse_workspace_id":\s*"([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})"
replace_value:
test: "$workspace.$id"
prod: "$workspace.$id"
# Optional field: Set to "true" to treat find_value as a regex pattern
is_regex: "true" # "<true|True>"
item_type: "Notebook" # filter on notebook filesIs there an ideal solution?
The replacement values, e.g. the Lakehouse ID don't change during the deployment. Ideally, the could be cached so that the API will be called only once for every replacement value.
Additional context, screenshots, logs, error output, etc
The full log (anonymized) is located at https://gist.github.com/MiSchroe/533fabeca5cbdf6312432b598deb21b3.
Currently, we have split the deployment into separate parts to stay in under one hour. Ideally, I would like to deploy all Fabric items in one deployment job.