[Teams Bot] add a lro evaluation test case

tools/sdk-ai-bots/azure-sdk-qa-bot-evaluation/tests/typespec_lro.jsonl

@@ -0,0 +1 @@
+{"testcase": "LRO header return 200", "query": "title: LRO header return 200\n\nquestion: Hi, I'm from the servicefabric RP team working with [azure-rest-api-specs/specification/servicefabricmanagedclusters/resource-manager/Microsoft.ServiceF…](https://github.com/Azure/azure-rest-api-specs/blob/main/specification/servicefabricmanagedclusters/resource-manager/Microsoft.ServiceFabric/ServiceFabricManagedClusters/preview/2025-06-01-preview/servicefabricmanagedclusters.json)\nOn our operations for ManagedClusters_CreateOrUpdate, NodeTypes_CreateOrUpdate, and ApplicationTypeVersions_CreateOrUpdate, we have both a 200 and 202 response defined.\nA 200 response is returned when the customer sends the initial request to create the resource. A 202 is returned on every subsequent PUT.\nI recently noticed that our service code returns an async operation URI in the 200 response for the above operations. https://msazure.visualstudio.com/One/_git/winfab-RP?path=/src/sfmc/SfmcBackendService/Service/Controllers/ClustersController.cs&version=GBdevelop&line=294&lineEnd=295&lineStartColumn=1&lineEndColumn=1&lineStyle=plain&_a=contents\n```\nHTTP/1.1 200 OK\nContent-length: 2131\nContent-Type: application/json; charset=utf-8\nServer: Microsoft-HTTPAPI/2.0\nAzure-AsyncOperation: http://localhost:8080/subscriptions/b36cdf46-b75d-4dc2-9fe1-1296ee8c623d/providers/Microsoft.ServiceFabric/locations/southcentralus/managedclusteroperations/c39cd8e1-18a0-41fc-a778-6a4ad6adbd653?api-version=2024-02-01\nDate: Thu, 23 Oct 2025 22:57:10 GMT\nConnection: close\n```\nShould I specify this in the spec? If so, what is the recommendation for doing so? I couldn't find an example for a 200 response with an async operation header.", "ground_truth": "You should NOT specify an Azure-AsyncOperation or Location header in the 200 response for resource creation (PUT) in your Azure REST API spec. According to the Azure ARM RPC and REST API guidelines, the correct behavior for long-running resource creation operations is:\n- On initial PUT for resource creation: Return 201 (Created) with the resource in the response body. If the operation is asynchronous, include the Azure-AsyncOperation header in the 201 response (not 200), and/or Location header as appropriate.\n- On subsequent PUTs (resource replacement): Return 200 (OK) with the resource in the response body. If the operation is asynchronous, include the Azure-AsyncOperation header in the 200 response (not 201), and/or Location header as appropriate.\nHowever, in both cases, the async operation headers (Azure-AsyncOperation and/or Location) should only be present if the provisioning is NOT complete (i.e., if the operation is truly asynchronous). The ARM guidelines do not support returning 200 with async headers for initial resource creation.\nIf your service returns 200 with an Azure-AsyncOperation header on initial resource creation, this is a violation of the ARM RPC and REST API guidelines. The correct approach is to return 201 with the async operation headers. See these references:\n- \"Respond to the initial Put request to create the resource with a 201 (Created). Respond to the subsequent Put request to replace the resource with a 200 (OK). Since provisioning is not complete, the Put response body must contain a provisioningState property. ... Starting January 2025, new Resource Provider namespace implementations must include the Azure-AsyncOperation header in their async responses.\" (azure_resource_manager_rpc)\n- \"Service returns the Azure-AsyncOperation header on the initial request and the final state is determined via that Azure-AsyncOperation endpoint. This represents a create-or-replace long-running operation for an Order resource; after the operation completes, a final GET is performed on the resource URL to obtain the current resource state.\" (typespec_azure_http_specs)\n- Examples in typespec_azure_http_specs and static_typespec_to_swagger_mapping show the Azure-AsyncOperation header in the 201 response for PUT create, and in the 200 response for PUT update, not in the 200 for initial create.\nBest practice:\n- For initial PUT (create): Only specify Azure-AsyncOperation/Location headers in the 201 response, not 200.\n- For subsequent PUT (update/replace): Specify Azure-AsyncOperation/Location headers in the 200 response, only if the operation is async.\nIf your implementation currently returns 200 + async operation header for initial create, you should update your service to comply with the guidelines and change it to 201 + async header for initial creation, and 200 + async header for replacement.\nHow to document in OpenAPI/TypeSpec: Here is a recommended TypeSpec pattern:\n`op createOrUpdate is ArmResourceCreateOrReplaceAsync<Resource>;`\nThis will generate Swagger like:\n```\n\"201\": {\n\"description\": \"Resource create operation succeeded\",\n\"schema\": { \"$ref\": \"#/definitions/Resource\" },\n\"headers\": {\n\"Azure-AsyncOperation\": { \"type\": \"string\", \"description\": \"A link to the status monitor\" },\n\"Retry-After\": { \"type\": \"integer\", \"format\": \"int32\", \"description\": \"How long to wait before polling\" }\n}\n},\n\"200\": {\n\"description\": \"Resource update operation succeeded\",\n\"schema\": { \"$ref\": \"#/definitions/Resource\" },\n\"headers\": {\n\"Azure-AsyncOperation\": { \"type\": \"string\", \"description\": \"A link to the status monitor\" },\n\"Retry-After\": { \"type\": \"integer\", \"format\": \"int32\", \"description\": \"How long to wait before polling\" }\n}\n}\n```\nSummary:\n- Do NOT return Azure-AsyncOperation/Location header in 200 response for initial resource creation. Only return in 201 response for initial create and in 200 response for replacement if async.\n- If your implementation violates this, update it to comply with ARM guidelines. Document your spec as above.", "context": "", "response_length": 4004}