[Langchain] 이미지 분석

[Langchain] 이미지 분석

Code

import base64
import os.path
from io import BytesIO

from PIL import Image, ImageFile
from langchain_ollama import OllamaLLM

from langchain_demos.utils.dev import green, magenta


def convert_to_base64(image: ImageFile.ImageFile) -> str:
    buffered = BytesIO()
    image.save(buffered, format="JPEG")
    return base64.b64encode(buffered.getvalue()).decode("utf-8")


def summarize_image(base64_encoded: str) -> str:
    prompt = """
    You are an assistant tasked with summarizing images for retrieval.
    These summaries will be embedded and used to retrieve the raw image. 
    Give a concise summary of the image that is well optimized for retrieval.
    """.strip()

    llm = OllamaLLM(model="llava")
    return llm.bind(images=[base64_encoded]).invoke(prompt)


if __name__ == '__main__':
    image_filepaths = []

    for root, _, files in os.walk("../../../data"):
        image_filepaths = [
            os.path.join(root, file)
            for file in files
            if file.endswith(".png") or file.endswith(".jpg") or file.endswith(".jpeg")
        ]

    # image_filepaths = ["../../../data/image.png"]

    for filepath in image_filepaths:
        green(filepath)
        image_base64 = convert_to_base64(Image.open(filepath))
        summary = summarize_image(image_base64)
        magenta(summary)

../../../data/덴탈최종_vlcatt_kcnzad.jpg
 The image shows a package of dog food. It is labeled "Precision 5G" and has the logo "DogFooders" on it, indicating that this product is likely targeted towards dogs with high energy needs or specific dietary requirements. The packaging is predominantly white and purple, with text and logos in shades of blue, and there are pictures of a dog and a sheep on the package. The style of the image suggests it's a product photograph intended for marketing or an online store listing. 

../../../data/01.38838209.1.jpg
 The image is a social media post with a large, digitally altered photo of a man in the center. He appears to be an older individual with a mustache and glasses. The text overlay on the image is in Korean, and it seems to include a message or announcement. To the left side of the image, there's a smaller photo that appears to show another individual, but due to the resolution and size, few details are discernible. There is also a graphic element on the right with Korean text that includes what looks like a stylized face with eyes closed, conveying a peaceful or positive emotion. The overall style of the image suggests it is a personal message from one person to another. 

../../../data/image.png
 The image shows a golden-colored dog with a joyful expression, covered in snow. It is sitting outdoors with its tongue hanging out, seemingly enjoying the snowy weather. The background is slightly blurred but appears to be a residential area with a house visible.