Colcon build fails with UnicodeDecodeError for WstringArrays in Foxy, but successful in dashing #610
Description
Bug report
Required Info:
- Operating System:
- Windows 10, linux debian 10 and macOs BigSur
- Installation type:
- from source( ros2 Foxy )
- DDS implementation:
- default shipped with ros2 foxy
Steps to reproduce issue
In Foxy, do colcon build for ros2 custom message with Wstrings.msg as shown below:
Wstrings.msg
wstring wstring_value
wstring wstring_value_default1 "Hello world!"
wstring wstring_value_default2 "Hellö wörld!"
wstring WSTRING_CONST="Hello world!"
wstring<=22 bounded_wstring_value
wstring<=22 bounded_wstring_value_default1 "Hello world!"
wstring[3] array_of_wstrings
wstring[<=3] bounded_sequence_of_wstrings
wstring[] unbounded_sequence_of_wstrings
wstring<=20[<=10] wstring_bounded_var_array
wstring<=10[<=10] wstring_bounded_var_array_default ['Hellö1','Hellö2','Hellö3','Hellö4']
wstring[<=10] wstring_bounded_array
wstring[<=10] wstring_bounded_array_default ["Hellö wörld!","Hello world!",'Hellö wörld!']
wstring<=20[5] wstring_limit_var_array
wstring<=20[5] wstring_limit_var_array_default ["Hellö wörld!1","Hellö wörld!2",'Hellö wörld!3',"Hellö wörld!4",'Hellö wörld!5']
wstring<=20 bounded_variable
wstring<=20 bounded_variable_default 'Hellö wörld!1'
wstring<=20[] wstring_limit_variable_length_array
wstring<=20[] wstring_limit_variable_length_array_default ["Hellö wörld!1","Hellö wörld!2",'Hellö wörld!3']
Build fails with the following error:
Error processing idl file: D:\ROS_2\ros2-foxy-20201211-windows-release.amd64\ros2_msg\build\ros2_msg\rosidl_adapter\ros2_msg\msg\TestWstring.idl
Traceback (most recent call last):
File "D:/dds/Foxy_source/ros2_foxy/install/share/rosidl_generator_cpp/cmake/../../../lib/rosidl_generator_cpp/rosidl_generator_cpp", line 40, in <module>
sys.exit(main())
File "D:/dds/Foxy_source/ros2_foxy/install/share/rosidl_generator_cpp/cmake/../../../lib/rosidl_generator_cpp/rosidl_generator_cpp", line 35, in main
args.generator_arguments_file,
File "D:\dds\Foxy_source\ros2_foxy\install\Lib\site-packages\rosidl_generator_cpp\__init__.py", line 40, in generate_cpp
post_process_callback=prefix_with_bom_if_necessary)
File "D:\dds\Foxy_source\ros2_foxy\install\Lib\site-packages\rosidl_cmake\__init__.py", line 96, in generate_files
raise(e)
File "D:\dds\Foxy_source\ros2_foxy\install\Lib\site-packages\rosidl_cmake\__init__.py", line 74, in generate_files
idl_file = parse_idl_file(locator)
File "D:\dds\Foxy_source\ros2_foxy\install\Lib\site-packages\rosidl_parser\parser.py", line 62, in parse_idl_file
string = locator.get_absolute_path().read_text(encoding='utf-8')
File "C:\python37\lib\pathlib.py", line 1207, in read_text
return f.read()
File "C:\python37\lib\codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf6 in position 985: invalid start byte
Expected behavior
Build should be successful
Actual behavior
Build failed
Additional information
Build is successful in ros2 dashing but broken in foxy.
In the Wstrings.msg file, these are the fields which are actually causing the failure in foxy.
wstring<=10[<=10] wstring_bounded_var_array_default ['Hellö1','Hellö2','Hellö3','Hellö4']
wstring[<=10] wstring_bounded_array_default ["Hellö wörld!","Hello world!",'Hellö wörld!']
wstring<=20[5] wstring_limit_var_array_default ["Hellö wörld!1","Hellö wörld!2",'Hellö wörld!3',"Hellö wörld!4",'Hellö wörld!5']
wstring<=20[] wstring_limit_variable_length_array_default ["Hellö wörld!1","Hellö wörld!2",'Hellö wörld!3']
stdout_stderr.log
Also, looking at the ros2 foxy codebase, I see that tests are missing for WstringArrays but present for StringArrays: