If you are using Python 3.8.0 and have installed the pyxll-jupyter package using "pip install pyxll-jupyter", but are not seeing the Jupyter Notebook icon in Excel then the problem may be your version of Python.
The first thing to check is your PyXLL log file. If you see an error in there related to not being able to import "shiboken2" the the problem may be due to a bug in early versions of Python 3.8 that made it incompatible with the PySide2 build from pypi.org (the one installed when using "pip install").
You may see one of these two errors in your log file:
DLL load failed while importing shiboken2: The specified procedure could not be found.
PySide2/__init__.py: Unable to import shiboken2
If you see one of these errors then check your version of Python by running "python --version".
If your Python version is earlier than 3.8.4 then you should update to the latest version of Python 3.8.
To upgrade Python you will either need to download the latest vesion from https://python.org, or if you are using Anaconda you can use "conda search python" to search for available Python versions and then "conda install python=3.8.x" using the latest 3.8 version.