Create a computer vision resource

Enter the details

Go to vision studio

In another tab, go to the Microsoft Foundry and create a cognitive service resource

In the Vision Studio click on below

Select Cognitive Service resource

Go to the Azure Foundry Project and create a new prompt flow

Create a standard flow- mrb_image_analysis_prompt_flow

Keep only Input, Output and Add python

rename python block to image_analysis

Add the following code

import os

import json

import requests

from promptflow import tool

key = "YOUR_PRIMARY_KEY"

endpoint = "YOUR_CV_ENDPOINT"

@tool

def detect_faces(image_url: str) -> str:

    data = {

        "url": image_url

    }

    url = (

        str(endpoint)

        + "computervision/imageanalysis:analyze"

        + "?api-version=2024-02-01"

        + "&features=faces"

    )

    headers = {

        "Content-Type": "application/json",

        "Ocp-Apim-Subscription-Key": key

    }

    result = requests.post(url=url, headers=headers, json=data).json()

    face_details = ""

    if result.get("faces"):

        for face in result["faces"]:

            face_details += (

                f"Face detected at "

                f"x={face['boundingBox']['x']}, "

                f"y={face['boundingBox']['y']}, "

                f"width={face['boundingBox']['w']}, "

                f"height={face['boundingBox']['h']}\n"

            )

    return face_details

Copy the Keys and endpoint and replace it in the code above

Start computing session

After the compute session’s started, click on “Validate and parse Input” button for your Python component. Set the image url

Now we will generating an LLM component. Click on the “LLM” component in the headers area and name it as “Summarisation”. 

Add the prompt below

# system:

You will be provided with face detection results for an image analysed using Azure Computer Vision.

The information includes details about detected faces such as their presence and bounding box

coordinates.

Your task is to answer the user query based only on the provided face detection information.

Do not assume any additional details about the image beyond what is explicitly provided.

If the provided face detection information is not sufficient to answer the user query,

respond with:

"I am sorry but I don't have sufficient knowledge to answer your query"

# user:

user query: {{question}}

face detection details: {{face_details}}

Set the connection below. Click on Validate and Parse input

Set the input parameters

Name the output variable as “GPT_response” and set it to “Summarisation.output”. 

Now Run the flow

Output Below