Skip to content
Antoine Boucher

Options

Profiles

← All articles
Article

ComfyUI-UML: diagrams as nodes, images and shareable source

2 min read

Listen to the audio story

Diagrams as nodes in ComfyUI · 2:12 · AI-written script, synthetic voices

Transcript

Ava: A screenshot of a diagram is a dead end. Nobody can fix a typo in it, compare it to last week's version or regenerate it.

Andrew: True. You end up redrawing the whole thing.

Ava: That's the itch behind ComfyUI-UML, a project Antoine wrote up. ComfyUI is known for image generation, but any intermediate result can be a node output. Here, diagram text goes in, and an image, a saved file and a shareable link come out.

Andrew: So the diagram source stays in the workflow.

Ava: Exactly. Mermaid or PlantUML source is easy to inspect and regenerate, and it travels with the rest of the workflow instead of becoming an unexplained picture. The README documents twenty-eight diagram types through Kroki, with output formats validated per type.

Andrew: Does it need an AI model to write the diagrams?

Ava: No, that's optional. But if a model does write one, the node cleans it up. It handles several response shapes, pulls out fenced code blocks and prefers one matching the selected diagram type. Useful when the model wraps prose around the Mermaid code.

Andrew: So it checks the syntax too?

Ava: No, and Antoine flags that. Extracting a clean block isn't validating it. The renderer still has to accept the diagram.

Andrew: What about getting an image back into ComfyUI?

Ava: Raster images go through Pillow, get converted to RGB, scaled between zero and one, and wrapped as a tensor with a batch of one. SVG is trickier. The code tries a CairoSVG conversion, and if that fails it returns a tiny placeholder while still keeping the saved file. So a saved SVG and its preview can differ.

Andrew: How would I start?

Ava: Install it with the comfy node command, restart ComfyUI, and load the single-diagram example. Two nodes, the diagram node into a preview. Add the viewer once rendering works.

Andrew: Any privacy catch?

Ava: The web backend sends your diagram source to the chosen Kroki service. For sensitive architecture or offline use, local Mermaid and Graphviz paths are documented. The full post has the flow diagram and links to the code.

A ComfyUI-UML diagram workflow, from the repository README
Table of contents

ComfyUI is useful beyond image generation when an intermediate result can become a node output. ComfyUI-UML applies that idea to diagrams: text enters a rendering node; an image, a saved file, and a shareable URL can leave it.

1. Keep the source in the workflow

A screenshot is hard to revise. Mermaid or PlantUML source is much easier to inspect, compare, and regenerate. The node lets that source travel with the rest of a ComfyUI workflow instead of becoming an unexplained image.

The README documents 28 Kroki diagram types, with output formats validated per type. This does not mean every renderer supports every format.

flowchart TD
  T["Diagram text or optional LLM output"] --> N["Normalize diagram source"]
  N --> U["UMLDiagram node"]
  U --> K["Kroki web renderer"]
  U --> L["Optional local Mermaid / Graphviz"]
  K --> O["Image, file and URL outputs"]
  L --> O
  O --> P["PreviewImage or Diagram Viewer URL"]

This is a conceptual view of the documented nodes, not a requirement to use an LLM.

2. What the code does with model output

nodes/uml_diagram.py[1] normalizes strings and several response shapes, extracts fenced diagram blocks, and prefers a block matching the selected type. That helps when a model returns prose around Mermaid code.

Extraction is not the same thing as syntax validation. A clean code block can still contain a broken diagram; the renderer must accept it.

3. From rendered bytes to a ComfyUI image

Raster output is opened with Pillow, converted to RGB, normalized to floating-point values between zero and one, and wrapped as a tensor shaped [1, H, W, 3].

SVG is different. The code attempts a CairoSVG conversion for an image preview. For non-raster output without a successful conversion, it can return a tiny placeholder tensor while preserving the file output. That distinction explains why a saved SVG and its preview may behave differently.

4. Start with the smallest workflow

comfy node install comfyui-uml

Then restart ComfyUI and try the repository’s uml_single_diagram_only.json: UMLDiagram → PreviewImage. Add the viewer after rendering works. The repository also supplies Mermaid, PlantUML, and Ollama examples.

The web backend sends diagram source to the selected Kroki service. For sensitive architecture or offline use, the repository documents optional local Mermaid and Graphviz paths. Local Mermaid needs its JavaScript dependencies; SVG preview may need CairoSVG.

5. Source trail

References

  1. antoinebou12. (). nodes/uml_diagram.py. [Source file or documentation] github.com.
  2. antoinebou12. (). ComfyUI-UML. [Software repository] github.com.
  3. antoinebou12. (). Example workflows. [Source file or documentation] github.com.

References use this article’s publication date.