Skip to content

Fix encoding error in shellscript on Windows - #4720

Open
alejoe91 wants to merge 2 commits into
SpikeInterface:mainfrom
alejoe91:fix-encoding-windows
Open

Fix encoding error in shellscript on Windows#4720
alejoe91 wants to merge 2 commits into
SpikeInterface:mainfrom
alejoe91:fix-encoding-windows

Conversation

@alejoe91

Copy link
Copy Markdown
Member

When checking installed sorters on Windows, I get this error:

---------------------------------------------------------------------------
UnicodeDecodeError                        Traceback (most recent call last)
Cell In[10], line 1
----> 1 ss.installed_sorters()

File [~\Documents\Codes\spikeinterface\src\spikeinterface\sorters\sorterlist.py:84](file:///C:/Users/alejo/Documents/Codes/spikeinterface/src/spikeinterface/sorters/sorterlist.py#line=83), in installed_sorters()
     81 def installed_sorters():
     82     """Lists installed sorters."""
---> 84     return sorted([s.sorter_name for s in sorter_full_list if s.is_installed()])

File [~\Documents\Codes\spikeinterface\src\spikeinterface\sorters\external\hdsort.py:86](file:///C:/Users/alejo/Documents/Codes/spikeinterface/src/spikeinterface/sorters/external/hdsort.py#line=85), in HDSortSorter.is_installed(cls)
     84 @classmethod
     85 def is_installed(cls):
---> 86     if cls.check_compiled():
     87         return True
     88     return check_if_installed(cls.hdsort_path)

File [~\Documents\Codes\spikeinterface\src\spikeinterface\sorters\basesorter.py:380](file:///C:/Users/alejo/Documents/Codes/spikeinterface/src/spikeinterface/sorters/basesorter.py#line=379), in BaseSorter.check_compiled(cls)
    372 shell_cmd = f"""
    373 #!/bin/bash
    374 if ! [ -x "$(command -v {cls.compiled_name})" ]; then
   (...)    377 fi
    378 """
    379 shell_script = ShellScript(shell_cmd)
--> 380 shell_script.start()
    381 retcode = shell_script.wait()
    382 if retcode != 0:

File [~\Documents\Codes\spikeinterface\src\spikeinterface\sorters\utils\shellscript.py:90](file:///C:/Users/alejo/Documents/Codes/spikeinterface/src/spikeinterface/sorters/utils/shellscript.py#line=89), in ShellScript.start(self)
     86 self._process = subprocess.Popen(
     87     cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, bufsize=1, universal_newlines=True, encoding=encoding
     88 )
     89 with open(script_log_path, "w+") as script_log_file:
---> 90     for line in self._process.stdout:
     91         script_log_file.write(line)
     92         if self._verbose:
     93             # Print onto console depending on the verbose property passed on from the sorter class

File <frozen codecs>:325, in BufferedIncrementalDecoder.decode(self, input, final)
    322 'Could not get source, probably due dynamically evaluated source code.'

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8a in position 9: invalid start byte

The fix is to ignore these errors :)

@alejoe91 alejoe91 added the sorters Related to sorters module label Jul 29, 2026
@zm711

zm711 commented Jul 29, 2026

Copy link
Copy Markdown
Member

Which terminal are you using @alejoe91 ? I haven't tested recent SI versions on my Windows computer, but old versions never had any issues (tested on Windows 10 and 11) so I'm curious what has changed?

@alejoe91

Copy link
Copy Markdown
Member Author

This happened in Windows 11, using jupyter lab via uv from a comman prompt. Seems related to some UTF updates on the Windows side

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sorters Related to sorters module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants