# ViT problem with GPU usage require image to be numpy **URL:** \ **Category:** 🤗Transformers\ **Created:** [June 3, 2022, 7:56am UTC](https://discuss.huggingface.co/t/vit-problem-with-gpu-usage-require-image-to-be-numpy/18678 "2022-06-03T07:56:17Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![marcomameli01](https://avatars.discourse-cdn.com/v4/letter/m/71e660/32.png) [@marcomameli01](https://discuss.huggingface.co/u/marcomameli01)\ **Post date:** [June 3, 2022, 7:56am UTC](https://discuss.huggingface.co/t/vit-problem-with-gpu-usage-require-image-to-be-numpy/18678/1 "2022-06-03T07:56:17Z")
Hello, I’m using ViT from your packages, in particular I’m using facebook/deit-base-patch16-384, when I try and test it on CPU all the workflow goes well but when I test it on GPU I receive this error: Using GPU Trainable param: 105521444 Traceback (most recent call last): File “D:\WorkingDirectory\UNIVPM\Progetti\ComputerGraphics\3DGen\myversion\pixel2mesh-geometric\p2m\train.py”, line 97, in pred\_points = model\_gcn(graph, pool) File “D:\WorkingDirectory\UNIVPM\Progetti\ComputerGraphics\3DGen\myversion\pixel2mesh-geometric\p2m\venv\lib\site-packages\torch\nn\modules\module.py”, line 1110, in \_call\_impl return forward\_call(\*input, \*\*kwargs) File “D:\WorkingDirectory\UNIVPM\Progetti\ComputerGraphics\3DGen\myversion\pixel2mesh-geometric\p2m\model\mesh\_network.py”, line 29, in forward features = pool(elli\_points, self.feat\_extr, self.transf) File “D:\WorkingDirectory\UNIVPM\Progetti\ComputerGraphics\3DGen\myversion\pixel2mesh-geometric\p2m\utils\pool.py”, line 16, in **call** feat\_conv3, feat\_conv4, feat\_conv5 = feat\_extr(self.im) File “D:\WorkingDirectory\UNIVPM\Progetti\ComputerGraphics\3DGen\myversion\pixel2mesh-geometric\p2m\venv\lib\site-packages\torch\nn\modules\module.py”, line 1110, in \_call\_impl return forward\_call(\*input, \*\*kwargs) File “D:\WorkingDirectory\UNIVPM\Progetti\ComputerGraphics\3DGen\myversion\pixel2mesh-geometric\p2m\model\image\transformer.py”, line 14, in forward inputs = self.feature\_extractor(x[0], return\_tensors=“pt”) File “D:\WorkingDirectory\UNIVPM\Progetti\ComputerGraphics\3DGen\myversion\pixel2mesh-geometric\p2m\venv\lib\site-packages\transformers\models\vit\feature\_extraction\_vit.py”, line 141, in **call** images = [self.resize(image=image, size=self.size, resample=self.resample) for image in images] File “D:\WorkingDirectory\UNIVPM\Progetti\ComputerGraphics\3DGen\myversion\pixel2mesh-geometric\p2m\venv\lib\site-packages\transformers\models\vit\feature\_extraction\_vit.py”, line 141, in images = [self.resize(image=image, size=self.size, resample=self.resample) for image in images] File “D:\WorkingDirectory\UNIVPM\Progetti\ComputerGraphics\3DGen\myversion\pixel2mesh-geometric\p2m\venv\lib\site-packages\transformers\image\_utils.py”, line 218, in resize image = self.to\_pil\_image(image) File “D:\WorkingDirectory\UNIVPM\Progetti\ComputerGraphics\3DGen\myversion\pixel2mesh-geometric\p2m\venv\lib\site-packages\transformers\image\_utils.py”, line 104, in to\_pil\_image image = image.numpy() TypeError: can’t convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first. I try to modify the image\_utils.py file at the line 104 with a changes that add .cpu().clone().numpy() but when I insert the changes the error becomes: Using GPU Trainable param: 105521444 Traceback (most recent call last): File “D:\WorkingDirectory\UNIVPM\Progetti\ComputerGraphics\3DGen\myversion\pixel2mesh-geometric\p2m\train.py”, line 97, in pred\_points = model\_gcn(graph, pool) File “D:\WorkingDirectory\UNIVPM\Progetti\ComputerGraphics\3DGen\myversion\pixel2mesh-geometric\p2m\venv\lib\site-packages\torch\nn\modules\module.py”, line 1110, in \_call\_impl return forward\_call(\*input, \*\*kwargs) File “D:\WorkingDirectory\UNIVPM\Progetti\ComputerGraphics\3DGen\myversion\pixel2mesh-geometric\p2m\model\mesh\_network.py”, line 29, in forward features = pool(elli\_points, self.feat\_extr, self.transf) File “D:\WorkingDirectory\UNIVPM\Progetti\ComputerGraphics\3DGen\myversion\pixel2mesh-geometric\p2m\utils\pool.py”, line 16, in **call** feat\_conv3, feat\_conv4, feat\_conv5 = feat\_extr(self.im) File “D:\WorkingDirectory\UNIVPM\Progetti\ComputerGraphics\3DGen\myversion\pixel2mesh-geometric\p2m\venv\lib\site-packages\torch\nn\modules\module.py”, line 1110, in \_call\_impl return forward\_call(\*input, \*\*kwargs) File “D:\WorkingDirectory\UNIVPM\Progetti\ComputerGraphics\3DGen\myversion\pixel2mesh-geometric\p2m\model\image\transformer.py”, line 16, in forward outputs = self.model(\*\*inputs, output\_hidden\_states=True) File “D:\WorkingDirectory\UNIVPM\Progetti\ComputerGraphics\3DGen\myversion\pixel2mesh-geometric\p2m\venv\lib\site-packages\torch\nn\modules\module.py”, line 1110, in \_call\_impl return forward\_call(\*input, \*\*kwargs) File “D:\WorkingDirectory\UNIVPM\Progetti\ComputerGraphics\3DGen\myversion\pixel2mesh-geometric\p2m\venv\lib\site-packages\transformers\models\vit\modeling\_vit.py”, line 572, in forward embedding\_output = self.embeddings( File “D:\WorkingDirectory\UNIVPM\Progetti\ComputerGraphics\3DGen\myversion\pixel2mesh-geometric\p2m\venv\lib\site-packages\torch\nn\modules\module.py”, line 1110, in \_call\_impl return forward\_call(\*input, \*\*kwargs) File “D:\WorkingDirectory\UNIVPM\Progetti\ComputerGraphics\3DGen\myversion\pixel2mesh-geometric\p2m\venv\lib\site-packages\transformers\models\vit\modeling\_vit.py”, line 135, in forward embeddings = self.patch\_embeddings(pixel\_values, interpolate\_pos\_encoding=interpolate\_pos\_encoding) File “D:\WorkingDirectory\UNIVPM\Progetti\ComputerGraphics\3DGen\myversion\pixel2mesh-geometric\p2m\venv\lib\site-packages\torch\nn\modules\module.py”, line 1110, in \_call\_impl return forward\_call(\*input, \*\*kwargs) File “D:\WorkingDirectory\UNIVPM\Progetti\ComputerGraphics\3DGen\myversion\pixel2mesh-geometric\p2m\venv\lib\site-packages\transformers\models\vit\modeling\_vit.py”, line 191, in forward x = self.projection(pixel\_values).flatten(2).transpose(1, 2) File “D:\WorkingDirectory\UNIVPM\Progetti\ComputerGraphics\3DGen\myversion\pixel2mesh-geometric\p2m\venv\lib\site-packages\torch\nn\modules\module.py”, line 1110, in \_call\_impl return forward\_call(\*input, \*\*kwargs) File “D:\WorkingDirectory\UNIVPM\Progetti\ComputerGraphics\3DGen\myversion\pixel2mesh-geometric\p2m\venv\lib\site-packages\torch\nn\modules\conv.py”, line 447, in forward return self.\_conv\_forward(input, self.weight, self.bias) File “D:\WorkingDirectory\UNIVPM\Progetti\ComputerGraphics\3DGen\myversion\pixel2mesh-geometric\p2m\venv\lib\site-packages\torch\nn\modules\conv.py”, line 443, in \_conv\_forward return F.conv2d(input, weight, bias, self.stride, RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same or input should be a MKLDNN tensor and weight is a dense tensor So I don’t understand why if the image that I use already a torch tensor do you need to convert it to numpy array() Can you help me please? ---
**Author:** ![marcomameli01](https://avatars.discourse-cdn.com/v4/letter/m/71e660/32.png) [@marcomameli01](https://discuss.huggingface.co/u/marcomameli01)\ **Post date:** [June 6, 2022, 7:18pm UTC](https://discuss.huggingface.co/t/vit-problem-with-gpu-usage-require-image-to-be-numpy/18678/2 "2022-06-06T19:18:26Z")
Please someone can help me? ---
**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\ **Post date:** [June 18, 2022, 9:08am UTC](https://discuss.huggingface.co/t/vit-problem-with-gpu-usage-require-image-to-be-numpy/18678/3 "2022-06-18T09:08:01Z")
Do you have a code snippet to reproduce your error? ---
**Author:** ![marcomameli01](https://avatars.discourse-cdn.com/v4/letter/m/71e660/32.png) [@marcomameli01](https://discuss.huggingface.co/u/marcomameli01)\ **Post date:** [June 24, 2022, 9:40pm UTC](https://discuss.huggingface.co/t/vit-problem-with-gpu-usage-require-image-to-be-numpy/18678/4 "2022-06-24T21:40:49Z")
I can share my github code, but privately for the moment and you can reproduce the environment with the data. If you agree please let me know your mail.