Merge pull request #409 from thygate/marigold

Two mode models supported

Author: semjon00

.gitignore

@@ -1,2 +1,3 @@
 __pycache__/
 venv/
+.idea/

CHANGELOG.md

@@ -1,4 +1,8 @@
 ## Changelog
+### 0.4.6
+ * Support for [Depth Anything](https://github.com/LiheYoung/Depth-Anything).
+### 0.4.5
+ * Preliminary support for [Marigold](https://marigoldmonodepth.github.io). [PR #385](https://github.com/thygate/stable-diffusion-webui-depthmap-script/pull/385).
 ### 0.4.4
  * Compatibility with stable-diffusion-webui 1.6.0
 ### 0.4.3 video processing tab

README.md

@@ -1,7 +1,7 @@
 # High Resolution Depth Maps for Stable Diffusion WebUI
 This program is an addon for [AUTOMATIC1111's Stable Diffusion WebUI](https://github.com/AUTOMATIC1111/stable-diffusion-webui) that creates depth maps. Using either generated or custom depth maps, it can also create 3D stereo image pairs (side-by-side or anaglyph), normalmaps and 3D meshes. The outputs of the script can be viewed directly or used as an asset for a 3D engine. Please see [wiki](https://github.com/thygate/stable-diffusion-webui-depthmap-script/wiki/Viewing-Results) to learn more. The program has integration with [Rembg](https://github.com/danielgatis/rembg). It also supports batch processing, processing of videos, and can also be run in standalone mode, without Stable Diffusion WebUI.
 
-To generate realistic depth maps from individual images, this script uses code and models from the [MiDaS](https://github.com/isl-org/MiDaS) and [ZoeDepth](https://github.com/isl-org/ZoeDepth) repositories by Intel ISL, or LeReS from the [AdelaiDepth](https://github.com/aim-uofa/AdelaiDepth) repository by Advanced Intelligent Machines. Multi-resolution merging as implemented by [BoostingMonocularDepth](https://github.com/compphoto/BoostingMonocularDepth) is used to generate high resolution depth maps.
+To generate realistic depth maps from individual images, this script uses code and models from the [Marigold](https://github.com/prs-eth/Marigold/) repository, from the [MiDaS](https://github.com/isl-org/MiDaS) and [ZoeDepth](https://github.com/isl-org/ZoeDepth) repositories by Intel ISL, or LeReS from the [AdelaiDepth](https://github.com/aim-uofa/AdelaiDepth) repository by Advanced Intelligent Machines. Multi-resolution merging as implemented by [BoostingMonocularDepth](https://github.com/compphoto/BoostingMonocularDepth) is used to generate high resolution depth maps.
 
 Stereoscopic images are created using a custom-written algorithm.
 
@@ -94,7 +94,7 @@ Feel free to comment and share in the discussions and submit issues.
 
 ## Acknowledgements
 
-This project relies on code and information from following papers : 
+This project relies on code and information from the following papers : 
 
 MiDaS :
 
@@ -198,3 +198,29 @@ ZoeDepth :
   copyright = {arXiv.org perpetual, non-exclusive license}
 }
 ```
+
+Marigold - Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation:
+
+```
+@misc{ke2023repurposing,
+      title={Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation}, 
+      author={Bingxin Ke and Anton Obukhov and Shengyu Huang and Nando Metzger and Rodrigo Caye Daudt and Konrad Schindler},
+      year={2023},
+      eprint={2312.02145},
+      archivePrefix={arXiv},
+      primaryClass={cs.CV}
+}
+```
+
+Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data
+
+```
+@misc{yang2024depth,
+      title={Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data}, 
+      author={Lihe Yang and Bingyi Kang and Zilong Huang and Xiaogang Xu and Jiashi Feng and Hengshuang Zhao},
+      year={2024},
+      eprint={2401.10891},
+      archivePrefix={arXiv},
+      primaryClass={cs.CV}
+}
+```

bundled_sources.txt

@@ -17,3 +17,6 @@ https://github.com/aim-uofa/AdelaiDepth/tree/main/LeReS/Minist_Test/lib/
 
 pix2pix
 https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix/
+
+Marigold
+https://github.com/prs-eth/Marigold/tree/22437a

install.py

@@ -1,6 +1,9 @@
+# Installs dependencies
+
 import launch
 import platform
 import sys
+import importlib.metadata
 
 # TODO: some dependencies apparently being reinstalled on every run. Investigate and fix.
 
@@ -38,6 +41,8 @@ def ensure(module_name, min_version=None):
     launch.run_pip('install "moviepy==1.0.2"', "moviepy requirement for depthmap script")
 ensure('transforms3d', '0.4.1')
 
+ensure('diffusers', '0.20.1')  # For Merigold
+
 ensure('imageio')  # 2.4.1
 try:  # Dirty hack to not reinstall every time
     importlib_metadata.version('imageio-ffmpeg')
@@ -52,3 +57,20 @@ def ensure(module_name, min_version=None):
 
 if platform.system() == 'Darwin':
     ensure('pyqt6')
+
+# Depth Anything
+def get_installed_version(package: str):
+    try:
+        return importlib.metadata.version(package)
+    except Exception:
+        return None
+def try_install_from_wheel(pkg_name: str, wheel_url: str):
+    if get_installed_version(pkg_name) is not None:
+        return
+    try:
+        launch.run_pip(f"install {wheel_url}", f" {pkg_name} requirement for depthmap script")
+    except Exception as e:
+        print('Failed to install wheel for Depth Anything support. It won\'t work.')
+try_install_from_wheel(
+    "depth_anything",
+    "https://github.com/huchenlei/Depth-Anything/releases/download/v1.0.0/depth_anything-2024.1.22.0-py2.py3-none-any.whl")

marigold/marigold/__init__.py

@@ -0,0 +1 @@
+from .marigold_pipeline import MarigoldPipeline, MarigoldDepthOutput