unicode errors with non-ascii characters in channel name

[jkpollard]

Been trying to play around with the code but my python knowledge is near zero.
Thanks for maintaining this script btw.
It was working great for me until recently I added some Irish language programs to my library.
The presence of any non-ascii characters in a channels name makes the script hang on startup as soon as it tries to create that channel, until I force restart Kodi.
For example a channel where the TV Network is "RTÉ2".
Or a channel set to a single TV Show, if there is an accented character in the title, same issue.
That same tv show title can be part of a channel, no errors, it only happens when the non-ascii character would appear in the channel name/title.
Here's the logs

2019-09-10 15:33:31.635 T:123145471717376 ERROR: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
- NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
Error Type: <type 'exceptions.UnicodeDecodeError'>
Error Contents: 'ascii' codec can't decode byte 0xc3 in position 2: ordinal not in range(128)
Traceback (most recent call last):
File "/Users//Library/Application Support/Kodi/addons/script.pseudotv/resources/lib/Overlay.py", line 179, in onInit
if self.readConfig() == False:
File "/Users//Library/Application Support/Kodi/addons/script.pseudotv/resources/lib/Overlay.py", line 259, in readConfig
self.channels = self.channelList.setupList()
File "/Users//Library/Application Support/Kodi/addons/script.pseudotv/resources/lib/ChannelList.py", line 110, in setupList
self.setupChannel(i + 1, False, makenewlists, False)
File "/Users//Library/Application Support/Kodi/addons/script.pseudotv/resources/lib/ChannelList.py", line 286, in setupChannel
if self.makeChannelList(channel, chtype, chsetting1, chsetting2, append) == True:
File "/Users//Library/Application Support/Kodi/addons/script.pseudotv/resources/lib/ChannelList.py", line 454, in makeChannelList
fle = self.makeTypePlaylist(chtype, setting1, setting2)
File "/Users//Library/Application Support/Kodi/addons/script.pseudotv/resources/lib/ChannelList.py", line 534, in makeTypePlaylist
return self.createNetworkPlaylist(setting1)
File "/Users/****/Library/Application Support/Kodi/addons/script.pseudotv/resources/lib/ChannelList.py", line 574, in createNetworkPlaylist
flename = xbmc.makeLegalFilename(GEN_CHAN_LOC + 'Network_' + network + '.xsp')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2: ordinal not in range(128)
-->End of Python script error report<--

I attempted adding some encode/decode methods but to no avail. I simply don't have the python knowledge to try figure it out.

[jkpollard]

Facing a similar error when trying to add those shows via a smart playlist instead:

2019-09-11 00:12:33.270 T:6480 ERROR: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
- NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
Error Type: <type 'exceptions.UnicodeEncodeError'>
Error Contents: 'ascii' codec can't encode character u'\xed' in position 105: ordinal not in range(128)
Traceback (most recent call last):
File "C:\Users_\AppData\Roaming\Kodi\addons\script.pseudotv\resources\lib\Overlay.py", line 179, in onInit
if self.readConfig() == False:
File "C:\Users_\AppData\Roaming\Kodi\addons\script.pseudotv\resources\lib\Overlay.py", line 259, in readConfig
self.channels = self.channelList.setupList()
File "C:\Users_\AppData\Roaming\Kodi\addons\script.pseudotv\resources\lib\ChannelList.py", line 110, in setupList
self.setupChannel(i + 1, False, makenewlists, False)
File "C:\Users_\AppData\Roaming\Kodi\addons\script.pseudotv\resources\lib\ChannelList.py", line 286, in setupChannel
if self.makeChannelList(channel, chtype, chsetting1, chsetting2, append) == True:
File "C:\Users_\AppData\Roaming\Kodi\addons\script.pseudotv\resources\lib\ChannelList.py", line 480, in makeChannelList
fileList = self.buildFileList(fle, channel)
File "C:\Users_\AppData\Roaming\Kodi\addons\script.pseudotv\resources\lib\ChannelList.py", line 1020, in buildFileList
fileList.extend(self.buildFileList(match.group(1), channel))
File "C:\Users_\AppData\Roaming\Kodi\addons\script.pseudotv\resources\lib\ChannelList.py", line 1007, in buildFileList
json_folder_detail = self.sendJSON(json_query)
File "C:\Users_\AppData\Roaming\Kodi\addons\script.pseudotv\resources\lib\ChannelList.py", line 173, in sendJSON
data = xbmc.executeJSONRPC(command)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xed' in position 105: ordinal not in range(128)
-->End of Python script error report<--

[rafaelvieiras]

I have the same problem! 😞

[Steveb1968]

Yep I hear ya. Unicode errors a bitch to catch! Especially with this code, its all over the place now. :)
Hopefully I'll fix it up for Py3

[rafaelvieiras]

I have a similar error! 😢

ERROR: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
                                             - NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
                                            Error Type: <type 'exceptions.UnicodeDecodeError'>
                                            Error Contents: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)
                                            Traceback (most recent call last):
                                              File "/home/rafael/.kodi/addons/script.pseudotv/resources/lib/Overlay.py", line 180, in onInit
                                                if self.readConfig() == False:
                                              File "/home/rafael/.kodi/addons/script.pseudotv/resources/lib/Overlay.py", line 261, in readConfig
                                                self.channels = self.channelList.setupList()
                                              File "/home/rafael/.kodi/addons/script.pseudotv/resources/lib/ChannelList.py", line 111, in setupList
                                                self.setupChannel(i + 1, False, makenewlists, False)
                                              File "/home/rafael/.kodi/addons/script.pseudotv/resources/lib/ChannelList.py", line 287, in setupChannel
                                                if self.makeChannelList(channel, chtype, chsetting1, chsetting2, append) == True:
                                              File "/home/rafael/.kodi/addons/script.pseudotv/resources/lib/ChannelList.py", line 455, in makeChannelList
                                                fle = self.makeTypePlaylist(chtype, setting1, setting2)
                                              File "/home/rafael/.kodi/addons/script.pseudotv/resources/lib/ChannelList.py", line 562, in makeTypePlaylist
                                                return self.createGenreMixedPlaylist(setting1)
                                              File "/home/rafael/.kodi/addons/script.pseudotv/resources/lib/ChannelList.py", line 637, in createGenreMixedPlaylist
                                                flename = xbmc.makeLegalFilename(GEN_CHAN_LOC + 'Mixed_' + genre + '.xsp')
                                            UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)
                                            -->End of Python script error report<--

[rafaelvieiras]

I made a fix and submited a pull request #38
I waiting aprove of @Steveb1968.

[Steveb1968]

Hi mate, yes I know about the..
reload(sys) sys.setdefaultencoding('utf-8')
I've used it before in testing. I have a commit to hopefully fix the encoding when writing files but I can pull this in as a stop gap.
I don't really want to use such a blanket sys mod like this but its ok until I get it fixed.

[Steveb1968]

@rafaelvieiras Hi mate, just pushed a few unicode changes to ChannelList.py.
If you could test them without the "sys.setdefaultencoding('utf-8')" that would be great.

[rafaelvieiras]

@Steveb1968 , I tried to run without setdefaultencoding, but, I receive the same error.

2019-11-16 01:15:12.841 T:139675826124544   ERROR: EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
                                             - NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
                                            Error Type: <type 'exceptions.UnicodeDecodeError'>
                                            Error Contents: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)
                                            Traceback (most recent call last):
                                              File "/home/x/.kodi/addons/script.pseudotv/resources/lib/Overlay.py", line 180, in onInit
                                                if self.readConfig() == False:
                                              File "/home/x/.kodi/addons/script.pseudotv/resources/lib/Overlay.py", line 262, in readConfig
                                                self.channels = self.channelList.setupList()
                                              File "/home/x/.kodi/addons/script.pseudotv/resources/lib/ChannelList.py", line 110, in setupList
                                                self.setupChannel(i + 1, False, makenewlists, False)
                                              File "/home/x/.kodi/addons/script.pseudotv/resources/lib/ChannelList.py", line 286, in setupChannel
                                                if self.makeChannelList(channel, chtype, chsetting1, chsetting2, append) == True:
                                              File "/home/x/.kodi/addons/script.pseudotv/resources/lib/ChannelList.py", line 454, in makeChannelList
                                                fle = self.makeTypePlaylist(chtype, setting1, setting2)
                                              File "/home/x/.kodi/addons/script.pseudotv/resources/lib/ChannelList.py", line 561, in makeTypePlaylist
                                                return self.createGenreMixedPlaylist(setting1)
                                              File "/home/x/.kodi/addons/script.pseudotv/resources/lib/ChannelList.py", line 636, in createGenreMixedPlaylist
                                                flename = xbmc.makeLegalFilename(GEN_CHAN_LOC + 'Mixed_' + genre + '.xsp')
                                            UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)
                                            -->End of Python script error report<--

OS: Ubuntu 19.04

[Steveb1968]

Yep, missed that one..
Try again. :)