nomadoor commited on
Commit
3bae5fb
·
verified ·
1 Parent(s): c9702ae

Upload 3 files

Browse files
Files changed (3) hide show
  1. README.md +56 -20
  2. README_ja.md +61 -0
  3. TRAINING_NOTES.md +24 -0
README.md CHANGED
@@ -1,27 +1,63 @@
1
- ---
2
- tags:
3
- - text-to-image
4
- - lora
5
- - diffusers
6
- - template:diffusion-lora
7
- widget:
8
- - output:
9
- url: images/ComfyUI_00193_.png
10
- text: '-'
11
- base_model: black-forest-labs/FLUX.2-klein-base-9B
12
- instance_prompt: null
13
- license: other
14
- license_name: flux-non-commercial-license
15
- license_link: >-
16
- https://huggingface.co/black-forest-labs/FLUX.2-klein-base-9B/blob/main/LICENSE.md
17
- ---
18
  # FLUX.2 Klein 9B 360 ERP Outpaint LoRA
19
 
20
- <Gallery />
 
 
21
 
 
22
 
 
23
 
24
- ## Download model
25
 
 
 
 
26
 
27
- [Download](/nomadoor/flux-2-klein-9B-360-erp-outpaint-lora/tree/main) them in the Files & versions tab.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  # FLUX.2 Klein 9B 360 ERP Outpaint LoRA
2
 
3
+ [![ComfyUI Node](https://img.shields.io/badge/ComfyUI-Panorama--Stickers-blue)](https://github.com/nomadoor/ComfyUI-Panorama-Stickers)
4
+ [![4B Model](https://img.shields.io/badge/Hugging%20Face-4B%20Model-green)](https://huggingface.co/nomadoor/flux-2-klein-4B-360-erp-outpaint-lora)
5
+ [![README JA](https://img.shields.io/badge/README-日本語-lightgrey)](./README_ja.md)
6
 
7
+ This is a LoRA for FLUX.2 Klein 9B that generates 360-degree panoramas.
8
 
9
+ It uses an approach where reference images are placed on an empty ERP canvas, and the surrounding area is filled by outpainting. This makes it possible to generate panorama images from a flexible layout.
10
 
11
+ ## Examples
12
 
13
+ | Input | Output |
14
+ | --- | --- |
15
+ | ![example input](https://i.gyazo.com/d2f8b13debee5e8ef733110f72ffe87a.png) | ![example output](https://i.gyazo.com/0ad707176d21bf6d6d2d02fbb141aa81.png) |
16
 
17
+ ## Usage
18
+
19
+ 1. Prepare a `2:1` ERP control image with projected reference images placed on a green canvas.
20
+ 2. Enter the trigger prompt below at the beginning of your prompt.
21
+ 3. Add any extra scene description after the trigger prompt.
22
+
23
+ Notes:
24
+
25
+ - The reference image is expected to be projected into ERP from a **pinhole view** based on its intended panorama placement.
26
+ - A UI for this workflow is available as a Gradio app / ComfyUI custom node.
27
+ - [nomadoor/ComfyUI-Panorama-Stickers](https://github.com/nomadoor/ComfyUI-Panorama-Stickers)
28
+
29
+ ## Trigger Prompt
30
+
31
+ ```text
32
+ Fill the green spaces according to the image. Outpaint as a seamless 360 equirectangular panorama (2:1). Keep the horizon level. Match left and right edges.
33
+ ```
34
+
35
+ ## Recommended Resolution
36
+
37
+ - Generation: `2048x1024` (2K ERP)
38
+ - For panorama use, 2K may still look coarse in some cases, so use upscaling if needed
39
+
40
+ ## Notes
41
+
42
+ - 🚨Distilled models do not produce good results in most cases. I am still looking for a solution, but for now you should use it with the base model.
43
+ - Most of the training data is realistic scenery, so non-photoreal domains may be less stable.
44
+ - Perfect seam matching is still difficult, and some outputs may benefit from post-processing such as inpainting.
45
+
46
+ ## Training Setup
47
+
48
+ - [TRAINING_NOTES.md](./TRAINING_NOTES.md)
49
+
50
+ ## Workflow
51
+
52
+ - `simple.json`
53
+ - `advanced.json (WIP)`
54
+
55
+ ## License
56
+
57
+ Please follow the license and usage terms of the base model [`black-forest-labs/FLUX.2-klein-base-9B`](https://huggingface.co/black-forest-labs/FLUX.2-klein-base-9B).
58
+ See `LICENSE` included in this folder for additional notes.
59
+
60
+ ## Support
61
+
62
+ I ended up spending a bit too much on GPU rentals 🫠
63
+ If you would like to support the project, I would really, really appreciate it if you use [☕Ko-fi](https://ko-fi.com/nomadoor) or [❤GitHub Sponsors](https://github.com/sponsors/nomadoor/).
README_ja.md ADDED
@@ -0,0 +1,61 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # FLUX.2 Klein 9B 360 ERP Outpaint LoRA
2
+
3
+ [![ComfyUI Node](https://img.shields.io/badge/ComfyUI-Panorama--Stickers-blue)](https://github.com/nomadoor/ComfyUI-Panorama-Stickers)
4
+ [![4B Model](https://img.shields.io/badge/Hugging%20Face-4B%20Model-green)](https://huggingface.co/nomadoor/flux-2-klein-4B-360-erp-outpaint-lora)
5
+ [![README EN](https://img.shields.io/badge/README-English-lightgrey)](./README.md)
6
+
7
+ 任意の画像から 360 度パノラマを生成する、FLUX.2 Klein 9B 向けの LoRA です。
8
+
9
+ 空の ERP キャンバス上に参照画像を配置し、周囲を outpainting で埋めるアプローチを採用しています。これにより、自由度の高いレイアウトからパノラマ画像を生成できます。
10
+
11
+ ## Examples
12
+
13
+ | Input | Output |
14
+ | --- | --- |
15
+ | ![example input](https://i.gyazo.com/d2f8b13debee5e8ef733110f72ffe87a.png) | ![example output](https://i.gyazo.com/0ad707176d21bf6d6d2d02fbb141aa81.png) |
16
+
17
+ ## 使い方
18
+
19
+ 1. `2:1` の ERP グリーンキャンバス上に、投影済みの参照画像を配置した `control` 画像を作成します。
20
+
21
+ * 参照画像は、パノラマ内での配置結果を **pinhole 視点から ERP に展開** したものを使用します。
22
+ * そのための UI(Gradio / ComfyUI カスタムノード)を用意しているので、ぜひ活用してください。
23
+ * [nomadoor/ComfyUI-Panorama-Stickers](https://github.com/nomadoor/ComfyUI-Panorama-Stickers)
24
+ 2. 下記のトリガープロンプトを **先頭** に入力します。
25
+ 3. 必要に応じて、その後ろに任意のプロンプトを追加してください。
26
+
27
+ ### トリガープロンプト
28
+
29
+ ```text
30
+ Fill the green spaces according to the image. Outpaint as a seamless 360 equirectangular panorama (2:1). Keep the horizon level. Match left and right edges.
31
+ ```
32
+
33
+ ## 推奨解像度
34
+
35
+ * 生成: `2048x1024`(2K ERP)
36
+ * パノラマ用途では 2K でも粗く見える場合があるため、必要に応じてアップスケールを併用してください。
37
+
38
+ ## 注意点
39
+
40
+ * 🚨Distilledモデルではほとんど良い結果が得られません。解決方法を探していますが、基本的にはbase modelと共に使用してください。
41
+ * 学習データのほとんどが実写風景なため、非実写ドメインでは安定しない可能性があります。
42
+ * 左右端(シーム)の完全一致は難しく、ケースによっては後処理(inpaint など)が必要です。
43
+
44
+ ## 学習設定
45
+
46
+ * [TRAINING_NOTES.md](./TRAINING_NOTES.md)
47
+
48
+ ## Workflow
49
+
50
+ * `simple.json`
51
+ * `advanced.json(WIP)`
52
+
53
+ ## ライセンス
54
+
55
+ この LoRA の利用条件は、ベースモデル [`black-forest-labs/FLUX.2-klein-base-9B`](https://huggingface.co/black-forest-labs/FLUX.2-klein-base-9B) のライセンス / 利用規約に従ってください。
56
+ 補足は同梱の `LICENSE` を参照してください。
57
+
58
+ ## Support
59
+
60
+ GPU のレンタルで、少し予算を使いすぎました🫠
61
+ もし応援していただけるなら、[☕Ko-fi](https://ko-fi.com/nomadoor) または [❤GitHub Sponsors](https://github.com/sponsors/nomadoor/) からサポートいただけるととてもとても嬉しいです。
TRAINING_NOTES.md ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Training Notes (9B)
2
+
3
+ ## Summary
4
+ - Base model: `FLUX.2-klein-base-9B`
5
+ - LoRA rank: `16`
6
+ - Optimizer: `AdamW8bit`
7
+ - Learning rate: `3e-5`
8
+ - Training steps (tentative best): `600`
9
+
10
+ ## Dataset
11
+ - Source images were collected from open-license panorama/HDRI archives on the web.
12
+ - License policy was centered on permissive sources (`CC0` / Public Domain), with part of the set including `CC BY` assets.
13
+ - After preprocessing and filtering, training was run on a curated ERP pair set of about `1000` samples.
14
+
15
+ ## Preprocess / Pair Generation
16
+ - Target format: ERP (equirectangular) `2:1`
17
+ - Source panoramas were normalized to ERP and cleaned (invalid aspect/failed reads removed).
18
+ - Control images were generated by:
19
+ - building a green ERP canvas,
20
+ - sampling `1-3` reference patches,
21
+ - projecting patches with a pinhole-based spherical mapping,
22
+ - handling seam continuity with wrap-aware placement.
23
+ - The model is trained to fill green regions while preserving visible reference context.
24
+