Computer vision in Foundry

Teach

This lesson introduces computer vision capabilities in Azure AI Foundry and when to use them in real solutions.

Start free lesson

• This is the free lesson for Module 6 (Foundry vision).

Computer vision capabilities in Foundry

• Image classification assigns one or more labels to an entire image (for example product category).
• Object detection locates and labels multiple objects with bounding boxes.
• Optical character recognition (OCR) reads printed and handwritten text from images.
• Spatial analysis interprets people and movement in a physical space from video input.
• Image tagging and captioning generate descriptive tags and natural-language descriptions.

Prebuilt vs custom vision models

• Use prebuilt vision models when your scenario matches common tasks (tags, OCR, common objects) and you want fast deployment.
• Use custom vision models when you need domain-specific labels (for example proprietary parts or branded products) and have labeled training images.
• A common approach is starting prebuilt for a baseline, then training a custom model only where quality gaps remain.

Typical REST and SDK call pattern

1. Create the Foundry vision resource and capture endpoint + key.
2. Choose the task endpoint (for example image analysis, OCR/read, or object detection).
3. Send an image (URL or binary) in the request payload.
4. Parse results — labels, bounding boxes, extracted text — with confidence scores.
5. Log outcomes and monitor for misclassification or low-confidence patterns.

Python-style pseudocode (concept)

endpoint = os.getenv("FOUNDRY_VISION_ENDPOINT")
api_key = os.getenv("FOUNDRY_VISION_KEY")

result = analyze_image(
    endpoint=endpoint,
    api_key=api_key,
    image_url="https://example.com/shelf.jpg",
    features=["tags", "objects", "read"],
)

for obj in result["objects"]:
    print(obj["label"], obj["confidence"], obj["boundingBox"])

for line in result["read"]["lines"]:
    print(line["text"])