Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PegasusTokenizer requires the SentencePiece library but it was not found in your environment #8963

Closed
marcoabrate opened this issue Dec 7, 2020 · 9 comments

Comments

@marcoabrate
Copy link

Environment info

  • transformers version: 4.0.0
  • Platform: Google Colab
  • Python version: 3.6.9

Who can help

tokenizers: @mfuntowicz
Pegasus: @patrickvonplaten

To reproduce

Steps to reproduce the behavior:

!pip install -U transformers
!pip install sentencepiece

from transformers import PegasusTokenizer
tokenizer = PegasusTokenizer.from_pretrained('google/pegasus-large')

Error:

ImportError                               Traceback (most recent call last)

<ipython-input-7-12d68b5e397b> in <module>()
      1 from transformers import PegasusTokenizer
----> 2 tokenizer =    PegasusTokenizer.from_pretrained('google/pegasus-large')

/usr/local/lib/python3.6/dist-packages/transformers/utils/dummy_sentencepiece_objects.py in from_pretrained(self, *args, **kwargs)
     54     @classmethod
     55     def from_pretrained(self, *args, **kwargs):
---> 56         requires_sentencepiece(self)
     57 
     58 

/usr/local/lib/python3.6/dist-packages/transformers/file_utils.py in requires_sentencepiece(obj)
    459     name = obj.__name__ if hasattr(obj, "__name__") else obj.__class__.__name__
    460     if not is_sentencepiece_available():
--> 461         raise ImportError(SENTENCEPIECE_IMPORT_ERROR.format(name))
    462 
    463 

ImportError: 
PegasusTokenizer requires the SentencePiece library but it was not found in your environment. Checkout the instructions on the
installation page of its repo: https://github.com/google/sentencepiece#installation and follow the ones
that match your environment.
@patrickvonplaten
Copy link
Contributor

patrickvonplaten commented Dec 7, 2020

Hey @marcoabrate,
I tried to reproduce your error in this colab without success: https://colab.research.google.com/drive/1nBCEtP773LplNodOSw5OBW-rJ84gizW5?usp=sharing can you check again ?

@marcoabrate
Copy link
Author

You are right @patrickvonplaten
To reproduce

!pip install -U transformers

from transformers import PegasusTokenizer
tokenizer = PegasusTokenizer.from_pretrained('google/pegasus-large')

!pip install sentencepiece

tokenizer = PegasusTokenizer.from_pretrained('google/pegasus-large')

Very weird

@mileschandler
Copy link

Having the same issue. Although solely in my Jupyter notebook. Running the code from a file is working fine..

@marcoabrate
Copy link
Author

Probably it's a way the Jupyter kernel works. Indeed, if you restart the kernel and install sentencepiece before is working.

@OneStromberg
Copy link

as @marcoabrate said, restated kernel at my project and without code changes everything started working

@Moshiii
Copy link

Moshiii commented Jan 5, 2021

I can confirm that this solution is still valid today. I encountered the same issue today.
I solved it by adding !pip install sentencepiece and then fully restart the Jupiter environment and rerun.

@SamitM1
Copy link

SamitM1 commented Jul 1, 2022

I am unable to install the SentencePiece library, this is the error i get when i do pip3 install sentencepiece:

error: legacy-install-failure

@rohansheelvant
Copy link

Restarting the kernel and using
!pip install Transformers==3.2.0 instead of !pip install Transformers, worked for me

@ramdeepak100
Copy link

After using !pip install sentencepiece just restart the kernel and run the cells. It will work fine

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants