Upload 3 files
Browse files- README.md +56 -20
- README_ja.md +61 -0
- TRAINING_NOTES.md +24 -0
README.md
CHANGED
|
@@ -1,27 +1,63 @@
|
|
| 1 |
-
---
|
| 2 |
-
tags:
|
| 3 |
-
- text-to-image
|
| 4 |
-
- lora
|
| 5 |
-
- diffusers
|
| 6 |
-
- template:diffusion-lora
|
| 7 |
-
widget:
|
| 8 |
-
- output:
|
| 9 |
-
url: images/ComfyUI_00193_.png
|
| 10 |
-
text: '-'
|
| 11 |
-
base_model: black-forest-labs/FLUX.2-klein-base-9B
|
| 12 |
-
instance_prompt: null
|
| 13 |
-
license: other
|
| 14 |
-
license_name: flux-non-commercial-license
|
| 15 |
-
license_link: >-
|
| 16 |
-
https://huggingface.co/black-forest-labs/FLUX.2-klein-base-9B/blob/main/LICENSE.md
|
| 17 |
-
---
|
| 18 |
# FLUX.2 Klein 9B 360 ERP Outpaint LoRA
|
| 19 |
|
| 20 |
-
|
|
|
|
|
|
|
| 21 |
|
|
|
|
| 22 |
|
|
|
|
| 23 |
|
| 24 |
-
##
|
| 25 |
|
|
|
|
|
|
|
|
|
|
| 26 |
|
| 27 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
# FLUX.2 Klein 9B 360 ERP Outpaint LoRA
|
| 2 |
|
| 3 |
+
[](https://github.com/nomadoor/ComfyUI-Panorama-Stickers)
|
| 4 |
+
[](https://huggingface.co/nomadoor/flux-2-klein-4B-360-erp-outpaint-lora)
|
| 5 |
+
[](./README_ja.md)
|
| 6 |
|
| 7 |
+
This is a LoRA for FLUX.2 Klein 9B that generates 360-degree panoramas.
|
| 8 |
|
| 9 |
+
It uses an approach where reference images are placed on an empty ERP canvas, and the surrounding area is filled by outpainting. This makes it possible to generate panorama images from a flexible layout.
|
| 10 |
|
| 11 |
+
## Examples
|
| 12 |
|
| 13 |
+
| Input | Output |
|
| 14 |
+
| --- | --- |
|
| 15 |
+
|  |  |
|
| 16 |
|
| 17 |
+
## Usage
|
| 18 |
+
|
| 19 |
+
1. Prepare a `2:1` ERP control image with projected reference images placed on a green canvas.
|
| 20 |
+
2. Enter the trigger prompt below at the beginning of your prompt.
|
| 21 |
+
3. Add any extra scene description after the trigger prompt.
|
| 22 |
+
|
| 23 |
+
Notes:
|
| 24 |
+
|
| 25 |
+
- The reference image is expected to be projected into ERP from a **pinhole view** based on its intended panorama placement.
|
| 26 |
+
- A UI for this workflow is available as a Gradio app / ComfyUI custom node.
|
| 27 |
+
- [nomadoor/ComfyUI-Panorama-Stickers](https://github.com/nomadoor/ComfyUI-Panorama-Stickers)
|
| 28 |
+
|
| 29 |
+
## Trigger Prompt
|
| 30 |
+
|
| 31 |
+
```text
|
| 32 |
+
Fill the green spaces according to the image. Outpaint as a seamless 360 equirectangular panorama (2:1). Keep the horizon level. Match left and right edges.
|
| 33 |
+
```
|
| 34 |
+
|
| 35 |
+
## Recommended Resolution
|
| 36 |
+
|
| 37 |
+
- Generation: `2048x1024` (2K ERP)
|
| 38 |
+
- For panorama use, 2K may still look coarse in some cases, so use upscaling if needed
|
| 39 |
+
|
| 40 |
+
## Notes
|
| 41 |
+
|
| 42 |
+
- 🚨Distilled models do not produce good results in most cases. I am still looking for a solution, but for now you should use it with the base model.
|
| 43 |
+
- Most of the training data is realistic scenery, so non-photoreal domains may be less stable.
|
| 44 |
+
- Perfect seam matching is still difficult, and some outputs may benefit from post-processing such as inpainting.
|
| 45 |
+
|
| 46 |
+
## Training Setup
|
| 47 |
+
|
| 48 |
+
- [TRAINING_NOTES.md](./TRAINING_NOTES.md)
|
| 49 |
+
|
| 50 |
+
## Workflow
|
| 51 |
+
|
| 52 |
+
- `simple.json`
|
| 53 |
+
- `advanced.json (WIP)`
|
| 54 |
+
|
| 55 |
+
## License
|
| 56 |
+
|
| 57 |
+
Please follow the license and usage terms of the base model [`black-forest-labs/FLUX.2-klein-base-9B`](https://huggingface.co/black-forest-labs/FLUX.2-klein-base-9B).
|
| 58 |
+
See `LICENSE` included in this folder for additional notes.
|
| 59 |
+
|
| 60 |
+
## Support
|
| 61 |
+
|
| 62 |
+
I ended up spending a bit too much on GPU rentals 🫠
|
| 63 |
+
If you would like to support the project, I would really, really appreciate it if you use [☕Ko-fi](https://ko-fi.com/nomadoor) or [❤GitHub Sponsors](https://github.com/sponsors/nomadoor/).
|
README_ja.md
ADDED
|
@@ -0,0 +1,61 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# FLUX.2 Klein 9B 360 ERP Outpaint LoRA
|
| 2 |
+
|
| 3 |
+
[](https://github.com/nomadoor/ComfyUI-Panorama-Stickers)
|
| 4 |
+
[](https://huggingface.co/nomadoor/flux-2-klein-4B-360-erp-outpaint-lora)
|
| 5 |
+
[](./README.md)
|
| 6 |
+
|
| 7 |
+
任意の画像から 360 度パノラマを生成する、FLUX.2 Klein 9B 向けの LoRA です。
|
| 8 |
+
|
| 9 |
+
空の ERP キャンバス上に参照画像を配置し、周囲を outpainting で埋めるアプローチを採用しています。これにより、自由度の高いレイアウトからパノラマ画像を生成できます。
|
| 10 |
+
|
| 11 |
+
## Examples
|
| 12 |
+
|
| 13 |
+
| Input | Output |
|
| 14 |
+
| --- | --- |
|
| 15 |
+
|  |  |
|
| 16 |
+
|
| 17 |
+
## 使い方
|
| 18 |
+
|
| 19 |
+
1. `2:1` の ERP グリーンキャンバス上に、投影済みの参照画像を配置した `control` 画像を作成します。
|
| 20 |
+
|
| 21 |
+
* 参照画像は、パノラマ内での配置結果を **pinhole 視点から ERP に展開** したものを使用します。
|
| 22 |
+
* そのための UI(Gradio / ComfyUI カスタムノード)を用意しているので、ぜひ活用してください。
|
| 23 |
+
* [nomadoor/ComfyUI-Panorama-Stickers](https://github.com/nomadoor/ComfyUI-Panorama-Stickers)
|
| 24 |
+
2. 下記のトリガープロンプトを **先頭** に入力します。
|
| 25 |
+
3. 必要に応じて、その後ろに任意のプロンプトを追加してください。
|
| 26 |
+
|
| 27 |
+
### トリガープロンプト
|
| 28 |
+
|
| 29 |
+
```text
|
| 30 |
+
Fill the green spaces according to the image. Outpaint as a seamless 360 equirectangular panorama (2:1). Keep the horizon level. Match left and right edges.
|
| 31 |
+
```
|
| 32 |
+
|
| 33 |
+
## 推奨解像度
|
| 34 |
+
|
| 35 |
+
* 生成: `2048x1024`(2K ERP)
|
| 36 |
+
* パノラマ用途では 2K でも粗く見える場合があるため、必要に応じてアップスケールを併用してください。
|
| 37 |
+
|
| 38 |
+
## 注意点
|
| 39 |
+
|
| 40 |
+
* 🚨Distilledモデルではほとんど良い結果が得られません。解決方法を探していますが、基本的にはbase modelと共に使用してください。
|
| 41 |
+
* 学習データのほとんどが実写風景なため、非実写ドメインでは安定しない可能性があります。
|
| 42 |
+
* 左右端(シーム)の完全一致は難しく、ケースによっては後処理(inpaint など)が必要です。
|
| 43 |
+
|
| 44 |
+
## 学習設定
|
| 45 |
+
|
| 46 |
+
* [TRAINING_NOTES.md](./TRAINING_NOTES.md)
|
| 47 |
+
|
| 48 |
+
## Workflow
|
| 49 |
+
|
| 50 |
+
* `simple.json`
|
| 51 |
+
* `advanced.json(WIP)`
|
| 52 |
+
|
| 53 |
+
## ライセンス
|
| 54 |
+
|
| 55 |
+
この LoRA の利用条件は、ベースモデル [`black-forest-labs/FLUX.2-klein-base-9B`](https://huggingface.co/black-forest-labs/FLUX.2-klein-base-9B) のライセンス / 利用規約に従ってください。
|
| 56 |
+
補足は同梱の `LICENSE` を参照してください。
|
| 57 |
+
|
| 58 |
+
## Support
|
| 59 |
+
|
| 60 |
+
GPU のレンタルで、少し予算を使いすぎました🫠
|
| 61 |
+
もし応援していただけるなら、[☕Ko-fi](https://ko-fi.com/nomadoor) または [❤GitHub Sponsors](https://github.com/sponsors/nomadoor/) からサポートいただけるととてもとても嬉しいです。
|
TRAINING_NOTES.md
ADDED
|
@@ -0,0 +1,24 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Training Notes (9B)
|
| 2 |
+
|
| 3 |
+
## Summary
|
| 4 |
+
- Base model: `FLUX.2-klein-base-9B`
|
| 5 |
+
- LoRA rank: `16`
|
| 6 |
+
- Optimizer: `AdamW8bit`
|
| 7 |
+
- Learning rate: `3e-5`
|
| 8 |
+
- Training steps (tentative best): `600`
|
| 9 |
+
|
| 10 |
+
## Dataset
|
| 11 |
+
- Source images were collected from open-license panorama/HDRI archives on the web.
|
| 12 |
+
- License policy was centered on permissive sources (`CC0` / Public Domain), with part of the set including `CC BY` assets.
|
| 13 |
+
- After preprocessing and filtering, training was run on a curated ERP pair set of about `1000` samples.
|
| 14 |
+
|
| 15 |
+
## Preprocess / Pair Generation
|
| 16 |
+
- Target format: ERP (equirectangular) `2:1`
|
| 17 |
+
- Source panoramas were normalized to ERP and cleaned (invalid aspect/failed reads removed).
|
| 18 |
+
- Control images were generated by:
|
| 19 |
+
- building a green ERP canvas,
|
| 20 |
+
- sampling `1-3` reference patches,
|
| 21 |
+
- projecting patches with a pinhole-based spherical mapping,
|
| 22 |
+
- handling seam continuity with wrap-aware placement.
|
| 23 |
+
- The model is trained to fill green regions while preserving visible reference context.
|
| 24 |
+
|