This project helps users to convert Documents (.pdf, .png, .jpg, .jpeg) into Markdown for you ease of ingestion into LLM workflows.
It uses a public LLM endpint (doc2md) here This endpoint simply gives images or pdfs (converted to images) to visual model and asks it to conver it into markdown.
Here is a quick snippet using python to perform such task:
# Client is your OpenAI compatible client model = 'meta-llama/Llama-3.2-11B-Vision-Instruct' prompt = "Convert the following image to just the markdown text, respond only with text and description of it if relevant." messages = [ { "role": "user", "content": [ { "type": "text", "text": prompt, }, { "type": "image_url", "image_url": { "url": f"{base64_image}" }, }, ] } ] response = client.chat.completions.create(model=model, messages=messages, stream=False, max_tokens=int(max_new_tokens), **kwargs)