Advertisement
kopyl

Untitled

Jul 23rd, 2023
102
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 28.65 KB | None | 0 0
  1. root@9f1677f0ab0c:/# ./start.sh
  2. Container Started
  3. Starting api
  4. starting worker
  5. credentials.py :1123 2023-07-24 01:11:56,173 Found credentials in environment variables.
  6. run handler – from 18 May 2023 02:08
  7. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbe950>: Failed to establish a new connection: [Errno 111] Connection refused')))
  8. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbca30>: Failed to establish a new connection: [Errno 111] Connection refused')))
  9. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbd360>: Failed to establish a new connection: [Errno 111] Connection refused')))
  10. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbc4c0>: Failed to establish a new connection: [Errno 111] Connection refused')))
  11. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbcb80>: Failed to establish a new connection: [Errno 111] Connection refused')))
  12. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbdbd0>: Failed to establish a new connection: [Errno 111] Connection refused')))
  13. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbdc90>: Failed to establish a new connection: [Errno 111] Connection refused')))
  14. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600d78b80>: Failed to establish a new connection: [Errno 111] Connection refused')))
  15. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbe320>: Failed to establish a new connection: [Errno 111] Connection refused')))
  16. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbda50>: Failed to establish a new connection: [Errno 111] Connection refused')))
  17. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbe6e0>: Failed to establish a new connection: [Errno 111] Connection refused')))
  18. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbc790>: Failed to establish a new connection: [Errno 111] Connection refused')))
  19. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbd1e0>: Failed to establish a new connection: [Errno 111] Connection refused')))
  20. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600d7b2e0>: Failed to establish a new connection: [Errno 111] Connection refused')))
  21. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbd420>: Failed to establish a new connection: [Errno 111] Connection refused')))
  22. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbc2b0>: Failed to establish a new connection: [Errno 111] Connection refused')))
  23. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbd540>: Failed to establish a new connection: [Errno 111] Connection refused')))
  24. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbdd80>: Failed to establish a new connection: [Errno 111] Connection refused')))
  25. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600d78d90>: Failed to establish a new connection: [Errno 111] Connection refused')))
  26. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbe500>: Failed to establish a new connection: [Errno 111] Connection refused')))
  27. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbd8a0>: Failed to establish a new connection: [Errno 111] Connection refused')))
  28. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbea40>: Failed to establish a new connection: [Errno 111] Connection refused')))
  29. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbc7f0>: Failed to establish a new connection: [Errno 111] Connection refused')))
  30. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbd030>: Failed to establish a new connection: [Errno 111] Connection refused')))
  31. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600d78ee0>: Failed to establish a new connection: [Errno 111] Connection refused')))
  32. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbd0f0>: Failed to establish a new connection: [Errno 111] Connection refused')))
  33. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbc1f0>: Failed to establish a new connection: [Errno 111] Connection refused')))
  34. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbcbb0>: Failed to establish a new connection: [Errno 111] Connection refused')))
  35. 2023-07-24 01:12:01,866 - ControlNet - INFO - ControlNet v1.1.233
  36. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbdc30>: Failed to establish a new connection: [Errno 111] Connection refused')))
  37. ControlNet preprocessor location: /workspace/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/downloads
  38. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbe020>: Failed to establish a new connection: [Errno 111] Connection refused')))
  39. 2023-07-24 01:12:02,150 - ControlNet - INFO - ControlNet v1.1.233
  40. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600d637f0>: Failed to establish a new connection: [Errno 111] Connection refused')))
  41. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbdde0>: Failed to establish a new connection: [Errno 111] Connection refused')))
  42. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbd480>: Failed to establish a new connection: [Errno 111] Connection refused')))
  43. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbc370>: Failed to establish a new connection: [Errno 111] Connection refused')))
  44. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbd0c0>: Failed to establish a new connection: [Errno 111] Connection refused')))
  45. preload_extensions_git_metadata for 8 extensions took 0.13s
  46. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbcca0>: Failed to establish a new connection: [Errno 111] Connection refused')))
  47. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600d63f10>: Failed to establish a new connection: [Errno 111] Connection refused')))
  48. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbccd0>: Failed to establish a new connection: [Errno 111] Connection refused')))
  49. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbc430>: Failed to establish a new connection: [Errno 111] Connection refused')))
  50. API is not available, retrying in 200ms... (HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /queue/status (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0600dbe9b0>: Failed to establish a new connection: [Errno 111] Connection refused')))
  51. run handler
  52. --- Starting Serverless Worker ---
  53. WARN | test_input set, using test_input as job input
  54. DEBUG | test_input_provided | Job Confirmed
  55. DEBUG | test_input_provided | Set Job ID
  56. INFO | test_input_provided | Started
  57. event: {'input': {'payload': {'prompt': '(Best quality, 8k, 32k, Masterpiece, UHD:1.2),Photo of Pretty Japanese woman, 1girl, with a huge diamond necklace, front view, with pink short ponytail, ((big warm grin)), wearing a white tulle suspender wedding dress with flower-shaped lace and pearl embellishments and a short sleeve, (medium short [dark brown} hair), (big breasts:1.4), double eyelid, front shot, stare into the distance', 'negative_prompt': 'sfw ng_deepnegative_v1_75t, ((badhandv4)), (worst quality:2), (low quality:2), (normal quality:2), lowres, bad anatomy, bad hands, normal quality, ((monochrome)), ((grayscale)) watermark, bad legs, bad arms', 'inference_steps': 20, 'cfg_scale': 7, 'seed': -1, 'enable_hr': True, 'hr_upscaler': 'Latent', 'sampler_name': 'DPM++ 2M Karras', 'denoising_strength': 0.7, 'width': 512, 'height': 512, 'n_iter': 1}, 'timestamp_request_sent': 0, 'model_name': 'v1-5-pruned-emaonly.safetensors'}, 'webhook': 'https://webhook.site/71233761-790c-4712-bde1-56cdcb5c0b4c', 'id': 'test_input_provided'}
  58. http://127.0.0.1:3000/sdapi/v1/options: 404 {"detail":"Not Found"}
  59. http://127.0.0.1:3000/sdapi/v1/options: 404 {"detail":"Not Found"}
  60. http://127.0.0.1:3000/sdapi/v1/options: 404 {"detail":"Not Found"}
  61. http://127.0.0.1:3000/sdapi/v1/options: 404 {"detail":"Not Found"}
  62. http://127.0.0.1:3000/sdapi/v1/options: 404 {"detail":"Not Found"}
  63. http://127.0.0.1:3000/sdapi/v1/options: 404 {"detail":"Not Found"}
  64. Running on local URL: http://127.0.0.1:3000
  65.  
  66. To create a public link, set `share=True` in `launch()`.
  67. Startup time: 8.8s (import torch: 1.4s, import gradio: 1.0s, import ldm: 1.0s, other imports: 1.3s, setup codeformer: 0.2s, api.py: 0.7s, controlnet.py: 0.3s, lora_script.py: 0.2s, create ui: 1.5s, gradio launch: 0.8s).
  68. http://127.0.0.1:3000/sdapi/v1/options: 200 null
  69. Elapsed time: 0.6852085590362549 seconds
  70. http://127.0.0.1:3000/sdapi/v1/refresh-checkpoints: 200 null
  71. Elapsed time: 0.35150909423828125 seconds
  72. Loading weights [6ce0161689] from /workspace/stable-diffusion-webui/models/Stable-diffusion/v1-5-pruned-emaonly.safetensors
  73. Creating model from config: /workspace/stable-diffusion-webui/configs/v1-inference.yaml
  74. LatentDiffusion: Running in eps-prediction mode
  75. DiffusionWrapper has 859.52 M params.
  76. Textual inversion embeddings loaded(0):
  77. Model loaded in 12.5s (create model: 0.4s, apply weights to model: 9.7s, apply half(): 0.5s, load VAE: 0.9s, move model to device: 0.3s, calculate empty prompt: 0.5s).
  78. http://127.0.0.1:3000/sdapi/v1/reload-checkpoint: 200 {}
  79. Elapsed time: 13.682453870773315 seconds
  80. 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:05<00:00, 8.39it/s]
  81. 0%| | 0/50 [00:00<?, ?it/s]
  82. *** API error: POST: http://127.0.0.1:3000/sdapi/v1/txt2img {'error': 'OutOfMemoryError', 'detail': '', 'body': '', 'errors': 'CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 23.65 GiB total capacity; 6.12 GiB already allocated; 7.93 GiB free; 14.12 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF'}
  83. Traceback (most recent call last):
  84. File "/usr/local/lib/python3.10/dist-packages/anyio/streams/memory.py", line 98, in receive
  85. return self.receive_nowait()
  86. File "/usr/local/lib/python3.10/dist-packages/anyio/streams/memory.py", line 93, in receive_nowait
  87. raise WouldBlock
  88. anyio.WouldBlock
  89.  
  90. During handling of the above exception, another exception occurred:
  91.  
  92. Traceback (most recent call last):
  93. File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/base.py", line 78, in call_next
  94. message = await recv_stream.receive()
  95. File "/usr/local/lib/python3.10/dist-packages/anyio/streams/memory.py", line 118, in receive
  96. raise EndOfStream
  97. anyio.EndOfStream
  98.  
  99. During handling of the above exception, another exception occurred:
  100.  
  101. Traceback (most recent call last):
  102. File "/workspace/stable-diffusion-webui/modules/api/api.py", line 153, in exception_handling
  103. return await call_next(request)
  104. File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/base.py", line 84, in call_next
  105. raise app_exc
  106. File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/base.py", line 70, in coro
  107. await self.app(scope, receive_or_disconnect, send_no_error)
  108. File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/base.py", line 108, in __call__
  109. response = await self.dispatch_func(request, call_next)
  110. File "/workspace/stable-diffusion-webui/modules/api/api.py", line 117, in log_and_time
  111. res: Response = await call_next(req)
  112. File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/base.py", line 84, in call_next
  113. raise app_exc
  114. File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/base.py", line 70, in coro
  115. await self.app(scope, receive_or_disconnect, send_no_error)
  116. File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/cors.py", line 83, in __call__
  117. await self.app(scope, receive, send)
  118. File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/gzip.py", line 24, in __call__
  119. await responder(scope, receive, send)
  120. File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/gzip.py", line 44, in __call__
  121. await self.app(scope, receive, self.send_with_gzip)
  122. File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/exceptions.py", line 79, in __call__
  123. raise exc
  124. File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/exceptions.py", line 68, in __call__
  125. await self.app(scope, receive, sender)
  126. File "/usr/local/lib/python3.10/dist-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
  127. raise e
  128. File "/usr/local/lib/python3.10/dist-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
  129. await self.app(scope, receive, send)
  130. File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 718, in __call__
  131. await route.handle(scope, receive, send)
  132. File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 276, in handle
  133. await self.app(scope, receive, send)
  134. File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 66, in app
  135. response = await func(request)
  136. File "/usr/local/lib/python3.10/dist-packages/fastapi/routing.py", line 273, in app
  137. raw_response = await run_endpoint_function(
  138. File "/usr/local/lib/python3.10/dist-packages/fastapi/routing.py", line 192, in run_endpoint_function
  139. return await run_in_threadpool(dependant.call, **values)
  140. File "/usr/local/lib/python3.10/dist-packages/starlette/concurrency.py", line 41, in run_in_threadpool
  141. return await anyio.to_thread.run_sync(func, *args)
  142. File "/usr/local/lib/python3.10/dist-packages/anyio/to_thread.py", line 33, in run_sync
  143. return await get_asynclib().run_sync_in_worker_thread(
  144. File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
  145. return await future
  146. File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 807, in run
  147. result = context.run(func, *args)
  148. File "/workspace/stable-diffusion-webui/modules/api/api.py", line 338, in text2imgapi
  149. processed = process_images(p)
  150. File "/workspace/stable-diffusion-webui/modules/processing.py", line 620, in process_images
  151. res = process_images_inner(p)
  152. File "/workspace/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/batch_hijack.py", line 42, in processing_process_images_hijack
  153. return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
  154. File "/workspace/stable-diffusion-webui/modules/processing.py", line 739, in process_images_inner
  155. samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
  156. File "/workspace/stable-diffusion-webui/modules/processing.py", line 1077, in sample
  157. samples = self.sampler.sample_img2img(self, samples, noise, self.hr_c, self.hr_uc, steps=self.hr_second_pass_steps or self.steps, image_conditioning=image_conditioning)
  158. File "/workspace/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 409, in sample_img2img
  159. samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
  160. File "/workspace/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 278, in launch_sampling
  161. return func()
  162. File "/workspace/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 409, in <lambda>
  163. samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
  164. File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
  165. return func(*args, **kwargs)
  166. File "/workspace/stable-diffusion-webui/repositories/k-diffusion/k_diffusion/sampling.py", line 594, in sample_dpmpp_2m
  167. denoised = model(x, sigmas[i] * s_in, **extra_args)
  168. File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
  169. return forward_call(*args, **kwargs)
  170. File "/workspace/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 177, in forward
  171. x_out[a:b] = self.inner_model(x_in[a:b], sigma_in[a:b], cond=make_condition_dict(c_crossattn, image_cond_in[a:b]))
  172. File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
  173. return forward_call(*args, **kwargs)
  174. File "/workspace/stable-diffusion-webui/repositories/k-diffusion/k_diffusion/external.py", line 112, in forward
  175. eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs)
  176. File "/workspace/stable-diffusion-webui/repositories/k-diffusion/k_diffusion/external.py", line 138, in get_eps
  177. return self.inner_model.apply_model(*args, **kwargs)
  178. File "/workspace/stable-diffusion-webui/modules/sd_hijack_utils.py", line 17, in <lambda>
  179. setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs))
  180. File "/workspace/stable-diffusion-webui/modules/sd_hijack_utils.py", line 28, in __call__
  181. return self.__orig_func(*args, **kwargs)
  182. File "/workspace/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 858, in apply_model
  183. x_recon = self.model(x_noisy, t, **cond)
  184. File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
  185. return forward_call(*args, **kwargs)
  186. File "/workspace/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 1335, in forward
  187. out = self.diffusion_model(x, t, context=cc)
  188. File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
  189. return forward_call(*args, **kwargs)
  190. File "/workspace/stable-diffusion-webui/modules/sd_unet.py", line 91, in UNetModel_forward
  191. return ldm.modules.diffusionmodules.openaimodel.copy_of_UNetModel_forward_for_webui(self, x, timesteps, context, *args, **kwargs)
  192. File "/workspace/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/modules/diffusionmodules/openaimodel.py", line 797, in forward
  193. h = module(h, emb, context)
  194. File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
  195. return forward_call(*args, **kwargs)
  196. File "/workspace/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/modules/diffusionmodules/openaimodel.py", line 84, in forward
  197. x = layer(x, context)
  198. File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
  199. return forward_call(*args, **kwargs)
  200. File "/workspace/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/modules/attention.py", line 334, in forward
  201. x = block(x, context=context[i])
  202. File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
  203. return forward_call(*args, **kwargs)
  204. File "/workspace/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/modules/attention.py", line 269, in forward
  205. return checkpoint(self._forward, (x, context), self.parameters(), self.checkpoint)
  206. File "/workspace/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/modules/diffusionmodules/util.py", line 121, in checkpoint
  207. return CheckpointFunction.apply(func, len(inputs), *args)
  208. File "/usr/local/lib/python3.10/dist-packages/torch/autograd/function.py", line 506, in apply
  209. return super().apply(*args, **kwargs) # type: ignore[misc]
  210. File "/workspace/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/modules/diffusionmodules/util.py", line 136, in forward
  211. output_tensors = ctx.run_function(*ctx.input_tensors)
  212. File "/workspace/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/modules/attention.py", line 272, in _forward
  213. x = self.attn1(self.norm1(x), context=context if self.disable_self_attn else None) + x
  214. File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
  215. return forward_call(*args, **kwargs)
  216. File "/workspace/stable-diffusion-webui/modules/hypernetworks/hypernetwork.py", line 412, in attention_CrossAttention_forward
  217. attn = sim.softmax(dim=-1)
  218. torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 23.65 GiB total capacity; 6.12 GiB already allocated; 7.93 GiB free; 14.12 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
  219.  
  220. ---
  221. DEBUG | test_input_provided | Handler output: {'error': 'AUTOMATIC_ERROR', 'statusCode': 500, 'body': '{"error":"OutOfMemoryError","detail":"","body":"","errors":"CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 23.65 GiB total capacity; 6.12 GiB already allocated; 7.93 GiB free; 14.12 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF"}'}
  222. DEBUG | test_input_provided | run_job return: {'error': 'AUTOMATIC_ERROR'}
  223. DEBUG | rp_debugger | Flag not set, skipping debugger output.
  224. WARN | Local test job results for test_input_provided: {"error": "AUTOMATIC_ERROR"}
  225. INFO | test_input_provided | Finished
  226. ERROR | Job test_input_provided failed with error: AUTOMATIC_ERROR
  227. root@9f1677f0ab0c:/# Connection to 100.65.14.54 closed.██████████████████████████████████████████ | 50/100 [00:50<00:04, 10.70it/s]
  228. Connection to ssh.runpod.io closed.
  229. Olehs-MacBook-Prod:Downloads olehkopyl$
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement