[Bug]: NotImplementedError: No operator found for `memory_efficient_attention_forward` with inputs: #7973

zhan-cn · 2023-02-21T02:19:27Z

Is there an existing issue for this?

I have searched the existing issues and checked the recent builds/commits

What happened?

when I run .\webui.bat --xformers or .\webui.bat --xformers --no-half --medvram,meet bug :
NotImplementedError: No operator found for memory_efficient_attention_forward with inputs:

Steps to reproduce the problem

1 .\webui.bat --xformers --no-half --medvram
2 login in http://127.0.0.1:7860/
3 choose jpg,then generate

What should have happened?

generate jgeg

Commit where the problem happens

.\webui.bat --xformers --no-half --medvram

What platforms do you use to access the UI ?

Windows

What browsers do you use to access the UI ?

Microsoft Edge

Command Line Arguments

no

List of extensions

no

Console logs

(venv) PS C:\stable-diffusion-webui> .\webui.bat --xformers
venv "C:\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Commit hash: 076d624a297532d6e4abebe5807fd7c7504d7a73
Installing requirements for Web UI
Launching Web UI with arguments: --xformers
Loading weights [d635794c1f] from C:\stable-diffusion-webui\models\Stable-diffusion\512-base-ema.ckpt
Creating model from config: C:\stable-diffusion-webui\models\Stable-diffusion\512-base-ema.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 865.91 M params.
Applying xformers cross attention optimization.
Textual inversion embeddings loaded(0):
Model loaded in 8.6s (load weights from disk: 3.3s, create model: 0.3s, apply weights to model: 1.0s, apply half(): 1.1s, move model to device: 0.9s, load textual inversion embeddings: 1.8s).
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Error completing request
Arguments: ('task(08ryrh9mj92xxtq)', 0, '', '', [], <PIL.Image.Image image mode=RGBA size=1099x1099 at 0x1F472FD9750>, None, None, None, None, None, None, 20, 0, 4, 0, 1, False, False, 1, 1, 7, 1.5, 0.75, -1.0, -1.0, 0, 0, 0, False, 512, 512, 0, 0, 32, 0, '', '', '', [], 0, '<ul>\n<li><code>CFG Scale</code> should be 2 or lower.</li>\n</ul>\n', True, True, '', '', True, 50, True, 1, 0, False, 4, 1, 'None', '<p style="margin-bottom:0.75em">Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8</p>', 128, 8, ['left', 'right', 'up', 'down'], 1, 0.05, 128, 4, 0, ['left', 'right', 'up', 'down'], False, False, 'positive', 'comma', 0, False, False, '', '<p style="margin-bottom:0.75em">Will upscale the image by the selected scale factor; use width and height sliders to set tile size</p>', 64, 0, 2, 1, '', 0, '', 0, '', True, False, False, False, 0) {}
Traceback (most recent call last):
  File "C:\stable-diffusion-webui\modules\call_queue.py", line 56, in f
    res = list(func(*args, **kwargs))
  File "C:\stable-diffusion-webui\modules\call_queue.py", line 37, in f
    res = func(*args, **kwargs)
  File "C:\stable-diffusion-webui\modules\img2img.py", line 171, in img2img
    processed = process_images(p)
  File "C:\stable-diffusion-webui\modules\processing.py", line 486, in process_images
    res = process_images_inner(p)
  File "C:\stable-diffusion-webui\modules\processing.py", line 577, in process_images_inner
    p.init(p.all_prompts, p.all_seeds, p.all_subseeds)
  File "C:\stable-diffusion-webui\modules\processing.py", line 1017, in init
    self.init_latent = self.sd_model.get_first_stage_encoding(self.sd_model.encode_first_stage(image))
  File "C:\stable-diffusion-webui\modules\sd_hijack_utils.py", line 17, in <lambda>
    setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs))
  File "C:\stable-diffusion-webui\modules\sd_hijack_utils.py", line 28, in __call__
    return self.__orig_func(*args, **kwargs)
  File "C:\stable-diffusion-webui\venv\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "C:\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 830, in encode_first_stage
    return self.first_stage_model.encode(x)
  File "C:\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\models\autoencoder.py", line 83, in encode
    h = self.encoder(x)
  File "C:\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\model.py", line 536, in forward
    h = self.mid.attn_1(h)
  File "C:\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\model.py", line 258, in forward
    out = xformers.ops.memory_efficient_attention(q, k, v, attn_bias=None, op=self.attention_op)
  File "C:\stable-diffusion-webui\venv\lib\site-packages\xformers\ops\fmha\__init__.py", line 196, in memory_efficient_attention
    return _memory_efficient_attention(
  File "C:\stable-diffusion-webui\venv\lib\site-packages\xformers\ops\fmha\__init__.py", line 292, in _memory_efficient_attention
    return _memory_efficient_attention_forward(
  File "C:\stable-diffusion-webui\venv\lib\site-packages\xformers\ops\fmha\__init__.py", line 308, in _memory_efficient_attention_forward
    op = _dispatch_fw(inp)
  File "C:\stable-diffusion-webui\venv\lib\site-packages\xformers\ops\fmha\dispatch.py", line 98, in _dispatch_fw
    return _run_priority_list(
  File "C:\stable-diffusion-webui\venv\lib\site-packages\xformers\ops\fmha\dispatch.py", line 73, in _run_priority_list
    raise NotImplementedError(msg)
NotImplementedError: No operator found for `memory_efficient_attention_forward` with inputs:
     query       : shape=(1, 4096, 1, 512) (torch.float16)
     key         : shape=(1, 4096, 1, 512) (torch.float16)
     value       : shape=(1, 4096, 1, 512) (torch.float16)
     attn_bias   : <class 'NoneType'>
     p           : 0.0
`cutlassF` is not supported because:
    xFormers wasn't build with CUDA support
`flshattF` is not supported because:
    xFormers wasn't build with CUDA support
    max(query.shape[-1] != value.shape[-1]) > 128
`tritonflashattF` is not supported because:
    xFormers wasn't build with CUDA support
    max(query.shape[-1] != value.shape[-1]) > 128
    triton is not available
    requires A100 GPU
`smallkF` is not supported because:
    xFormers wasn't build with CUDA support
    dtype=torch.float16 (supported: {torch.float32})
    max(query.shape[-1] != value.shape[-1]) > 32
    unsupported embed per head: 512

Additional information

have been rebulid xformers,I think maybe i use gtx 1650 4G

The text was updated successfully, but these errors were encountered:

Setmaster · 2023-02-22T07:08:22Z

Having the same issue, RTX 3090.

Neefay · 2023-02-22T16:39:58Z

Same here, RTX 3080.

PandaBearz · 2023-02-23T17:46:12Z

same

SeasonalFerret · 2023-02-28T00:41:13Z

Same issue, 1660-ti. I've tried using pip install, I've tried manually building, I've tried Xformers (windows installation, wiki), I've tried the xformers re-install argument, nothing's working.

SeasonalFerret · 2023-02-28T00:51:08Z

Problem appears to have been resolved by following the steps in #6871 (comment). Just delete venv folder and run webui with --xformers.

I've gone from getting 2.1s/it to 1.75s/it thanks to Xformers.

CreamyLong · 2023-03-05T12:40:59Z

Having the same issue, win10，RTX 3060，cuda11.1.

Natotela · 2023-03-19T23:43:11Z

appears to have been resolved by following the steps in #6871 (comment). Just delete venv folder and run webui with --xformers

I deleted the xformers site-packages from the venv and pip installed xformers==0.0.16
but when I ran the webui, it just installed the venv xformers back. xformers-0.0.16rc425.dist-info

Build cuda_11.8.r11.8/compiler.31833905_0
RTX 3070

zz412000428 · 2023-03-24T08:50:44Z

Same here, ubuntu18.04, RTX 3080 Ti, cuda12.1

zoezhu · 2023-03-24T10:21:22Z

I met this issue too, in my case, I pull new code and lauch webui, and new launch.py install xformer again, which version is xformers==0.0.16rc425 and not compiled with my cuda version, I just uninstall it and install xformer from source again, everything goes fine now. I also comment out the line run_pip(f"install -r \"{requirements_file}\"", "requirements for Web UI") in launch.py.

Natotela · 2023-03-24T12:06:47Z

Also a good advice would be to check everything goes on one python version. I had some difference between the PIP path which led to a 3.9v and the Py path which was 3.10

zz412000428 · 2023-03-26T09:01:29Z

@zoezhu God bless, it's successful

panta5 · 2023-03-29T01:25:53Z

I solved it by temporarily removing the --xformers flag.
I'm penalized in speed, but so what.

Emekaborisama · 2023-04-17T13:49:55Z

I solved it by temporarily removing the --xformers flag. I'm penalized in speed, but so what.

sorry what did you remove? can you elaborate

Natotela · 2023-04-18T13:30:49Z

args

when you run webui.bat you have flags, command arguments, such as --no-half, or in many cases --xformers
instructing the use of python lib xformers. so he launched it without the work of that lib.

jjangga0214 · 2023-04-27T08:52:22Z

This also happens with Apple Silicon(M1 max, Ventura 13.3.1(22E261))

olim-ibragimov · 2023-05-03T23:00:11Z

In my case, I set the version of xformers to 0.0.16rc425 in launch.py (line 228). And it seems to work.

nbollman · 2023-05-05T19:43:11Z

Im running the vladmandic/automatic version. Getting some requirements mismatches I had to adjust just to get the program to run, between versions for python/torch/torchvision/xformers intercompatibility. Getting similar error. Ubuntu 22.02 Ryzen5800x RTX3090

xformers installed: ubuntu-22.04-py3.10-torch2.0.0+cu118

Launching launch.py...
14:32:14-702100 INFO Starting SD.Next
14:32:14-704433 INFO Python 3.10.10 on Linux
14:32:14-724191 INFO Version: 99dc75c Fri May 5 09:28:44 2023 -0400
14:32:15-339630 INFO Latest published version: f6898c9 2023-05-05T17:40:53Z
14:32:15-341186 INFO Setting environment tuning
14:32:15-342024 INFO nVidia CUDA toolkit detected
14:32:16-086818 INFO Torch 2.0.1+cu118
14:32:16-096932 INFO Torch backend: nVidia CUDA 11.8 cuDNN 8700
14:32:16-107578 INFO Torch detected GPU: NVIDIA GeForce RTX 3090 VRAM 24257 Arch (8, 6) Cores 82

...Blah blah, xformers loaded, start to generate image (shows in preview) and this poops out...

``NotImplementedError: No operator found for memory_efficient_attention_forward with inputs:
query : shape=(1, 4096, 1, 512) (torch.float16)
key : shape=(1, 4096, 1, 512) (torch.float16)
value : shape=(1, 4096, 1, 512) (torch.float16)
attn_bias : <class 'NoneType'>
p : 0.0
`cutlassF` is not supported because:
xFormers wasn't build with CUDA support
Operator wasn't built - see `python -m xformers.info` for more info
`flshattF` is not supported because:
xFormers wasn't build with CUDA support
max(query.shape[-1] != value.shape[-1]) > 128
Operator wasn't built - see `python -m xformers.info` for more info
`tritonflashattF` is not supported because:
xFormers wasn't build with CUDA support
max(query.shape[-1] != value.shape[-1]) > 128
requires A100 GPU
`smallkF` is not supported because:
xFormers wasn't build with CUDA support
dtype=torch.float16 (supported: {torch.float32})
max(query.shape[-1] != value.shape[-1]) > 32
Operator wasn't built - see `python -m xformers.info` for more info
unsupported embed per head: 512

Guess I shoulda but the A00? Im gonna try and build it, but am not sure how to activate the correct python venv for the project...

YoGalen · 2023-05-07T07:47:30Z

Problem appears to have been resolved by following the steps in #6871 (comment). Just delete venv folder and run webui with --xformers.

I've gone from getting 2.1s/it to 1.75s/it thanks to Xformers.

hello, where is venv directory?

Natotela · 2023-05-07T11:01:40Z

hello, where is venv directory

look in the stable-diffusion-webui dir for "venv"

wangchaofan2018 · 2023-06-03T11:10:06Z

I try pip install -v -U git+https://github.com/facebookresearch/xformers.git@main#egg=xformers. It works!

al-swaiti · 2023-06-06T11:23:04Z

hi every one i fix the issue for LINUX USERS BY EDIT THIS FILE "stable-diffusion-webui/modules/launch_utils.py"
and edit this line "xformers_package" to latest xformers package as below >
xformers_package = os.environ.get('XFORMERS_PACKAGE', 'xformers==0.0.21.dev543')
and relaunch the "./webui.sh --xformers" also windows users can try this way try it before delete venv folder if it doesn't work rename or delete venv and relunch webui with xformers flag

mikikokato · 2023-10-20T07:18:47Z

How do I resolve this error? I uninstalled it and installed it again, but it doesn't solve the problem.

NotImplementedError: No operator found for memory_efficient_attention_forward with inputs: query : shape=(2, 4096, 8, 40) (torch.float16) key : shape=(2, 4096, 8, 40) (torch.float16) value : shape=(2, 4096, 8, 40) (torch.float16) attn_bias : <class 'NoneType'> p : 0.0 flshattF is not supported because: xFormers wasn't build with CUDA support Operator wasn't built - see python -m xformers.info for more info tritonflashattF is not supported because: xFormers wasn't build with CUDA support Operator wasn't built - see python -m xformers.info for more info triton is not available requires A100 GPU cutlassF is not supported because: xFormers wasn't build with CUDA support Operator wasn't built - see python -m xformers.info for more info smallkF is not supported because: xFormers wasn't build with CUDA support dtype=torch.float16 (supported: {torch.float32}) max(query.shape[-1] != value.shape[-1]) > 32 Operator wasn't built - see python -m xformers.info for more info unsupported embed per head: 40

wasa4587 · 2023-11-03T19:44:48Z

Same here, RTX 3090
windows 10

theprogrammerknownasat · 2023-11-12T20:47:22Z

I have the same problem, 7900xtx, Fedora Linux

axel578 · 2023-12-05T11:02:46Z

same prblem rtx 3090, the problem occured for me since the beginning of sdxl

yohyama0216 · 2023-12-15T14:34:30Z

I used Google Colab and tried !pip install --pre -U xformers.
it works!

elBlacksmith · 2023-12-15T23:53:29Z

I used Google Colab and tried !pip install --pre -U xformers. it works!

Thanks, that worked!

sibidibidi · 2024-01-08T08:45:11Z

Windows 11, gtx 1660s, r5 5600, 16gb, same problem

inchinet · 2024-01-15T14:53:15Z

win11, PC (not colab) install SD, same error, what is the solution ?
delete venv folder and run webui with --xformers ?

XYZ-916 · 2024-01-30T07:27:25Z

I uninstall xformers and then install it solving the problem.

Hemanthkumar2112 · 2024-04-19T13:24:35Z

worked in kaggle platform !pip install --pre -U xformers

Ashutoshgaruda · 2024-05-24T07:20:01Z

try this
pip3 install -U xformers --index-url https://download.pytorch.org/whl/cu118

feedsbrain · 2024-06-07T02:29:31Z

If anyone still has issues with xformers in MacOS, here is what I did:

Add --xformers in COMMANDLINE_ARGS in webui-user.sh
Delete venv with rm -rf venv
Run webui.sh with this command (using llvm from brew):

brew install llvm
CC=/usr/local/opt/llvm/bin/clang CXX=/usr/local/opt/llvm/bin/clang++ ./webui.sh

xformers will be installed on the webui.sh launch

However, I think this will not work without CUDA. I'm looking if there's any alternatives to make it work for MPS.

zhan-cn added the bug-report Report of a bug, yet to be confirmed label Feb 21, 2023

cyril23 mentioned this issue Nov 4, 2023

auto (AUTOMATIC1111): TypeError: AsyncConnectionPool.__init__() got an unexpected keyword argument 'socket_options' AbdBarho/stable-diffusion-webui-docker#608

Closed

2 tasks

elBlacksmith mentioned this issue Dec 16, 2023

temporal fix for the "NotImplementedError" error (Related with the rest of xformers problems?) TheLastBen/fast-stable-diffusion#2690

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: NotImplementedError: No operator found for `memory_efficient_attention_forward` with inputs: #7973

[Bug]: NotImplementedError: No operator found for `memory_efficient_attention_forward` with inputs: #7973

zhan-cn commented Feb 21, 2023

Setmaster commented Feb 22, 2023

Neefay commented Feb 22, 2023

PandaBearz commented Feb 23, 2023

SeasonalFerret commented Feb 28, 2023 •

edited

Loading

SeasonalFerret commented Feb 28, 2023 •

edited

Loading

CreamyLong commented Mar 5, 2023

Natotela commented Mar 19, 2023 •

edited

Loading

zz412000428 commented Mar 24, 2023

zoezhu commented Mar 24, 2023 •

edited

Loading

Natotela commented Mar 24, 2023

zz412000428 commented Mar 26, 2023

panta5 commented Mar 29, 2023

Emekaborisama commented Apr 17, 2023

Natotela commented Apr 18, 2023

jjangga0214 commented Apr 27, 2023

olim-ibragimov commented May 3, 2023 •

edited

Loading

nbollman commented May 5, 2023 •

edited

Loading

YoGalen commented May 7, 2023

Natotela commented May 7, 2023

wangchaofan2018 commented Jun 3, 2023

al-swaiti commented Jun 6, 2023

mikikokato commented Oct 20, 2023

wasa4587 commented Nov 3, 2023

theprogrammerknownasat commented Nov 12, 2023

axel578 commented Dec 5, 2023

yohyama0216 commented Dec 15, 2023

elBlacksmith commented Dec 15, 2023

sibidibidi commented Jan 8, 2024

inchinet commented Jan 15, 2024

XYZ-916 commented Jan 30, 2024

Hemanthkumar2112 commented Apr 19, 2024 •

edited

Loading

Ashutoshgaruda commented May 24, 2024

feedsbrain commented Jun 7, 2024 •

edited

Loading

[Bug]: NotImplementedError: No operator found for memory_efficient_attention_forward with inputs: #7973

[Bug]: NotImplementedError: No operator found for memory_efficient_attention_forward with inputs: #7973

Comments

zhan-cn commented Feb 21, 2023

Is there an existing issue for this?

What happened?

Steps to reproduce the problem

What should have happened?

Commit where the problem happens

What platforms do you use to access the UI ?

What browsers do you use to access the UI ?

Command Line Arguments

List of extensions

Console logs

Additional information

Setmaster commented Feb 22, 2023

Neefay commented Feb 22, 2023

PandaBearz commented Feb 23, 2023

SeasonalFerret commented Feb 28, 2023 • edited Loading

SeasonalFerret commented Feb 28, 2023 • edited Loading

CreamyLong commented Mar 5, 2023

Natotela commented Mar 19, 2023 • edited Loading

zz412000428 commented Mar 24, 2023

zoezhu commented Mar 24, 2023 • edited Loading

Natotela commented Mar 24, 2023

zz412000428 commented Mar 26, 2023

panta5 commented Mar 29, 2023

Emekaborisama commented Apr 17, 2023

Natotela commented Apr 18, 2023

jjangga0214 commented Apr 27, 2023

olim-ibragimov commented May 3, 2023 • edited Loading

nbollman commented May 5, 2023 • edited Loading

YoGalen commented May 7, 2023

Natotela commented May 7, 2023

wangchaofan2018 commented Jun 3, 2023

al-swaiti commented Jun 6, 2023

mikikokato commented Oct 20, 2023

wasa4587 commented Nov 3, 2023

theprogrammerknownasat commented Nov 12, 2023

axel578 commented Dec 5, 2023

yohyama0216 commented Dec 15, 2023

elBlacksmith commented Dec 15, 2023

sibidibidi commented Jan 8, 2024

inchinet commented Jan 15, 2024

XYZ-916 commented Jan 30, 2024

Hemanthkumar2112 commented Apr 19, 2024 • edited Loading

Ashutoshgaruda commented May 24, 2024

feedsbrain commented Jun 7, 2024 • edited Loading

[Bug]: NotImplementedError: No operator found for `memory_efficient_attention_forward` with inputs: #7973

[Bug]: NotImplementedError: No operator found for `memory_efficient_attention_forward` with inputs: #7973

SeasonalFerret commented Feb 28, 2023 •

edited

Loading

SeasonalFerret commented Feb 28, 2023 •

edited

Loading

Natotela commented Mar 19, 2023 •

edited

Loading

zoezhu commented Mar 24, 2023 •

edited

Loading

olim-ibragimov commented May 3, 2023 •

edited

Loading

nbollman commented May 5, 2023 •

edited

Loading

Hemanthkumar2112 commented Apr 19, 2024 •

edited

Loading

feedsbrain commented Jun 7, 2024 •

edited

Loading