Using the sed command in Singularity Exec does not work. Similar with Redirect ">" symbol.

[jjacobson95]

Command line works as expected:
singularity exec --writable-tmpfs instance://reclusive_cat_2780 sed -i 's/REPLACE_THIS_LINE/ <parameter name="Raw data file names"><file>\/Work\/III_mzML\/Dorrestein_GnPS_P14_G10_7V_Rep01_Neg_29Dec20_Fiji_Infusion_Min50_MA-csum-Min20_3.mzML<\/file><\/parameter>/' /Work/MZmine_FeatureFinder-batch.xml

Python Command doesn't work:

file_name = "test_file"
command_list_1 = """sed -i \'s/REPLACE_THIS_LINE/        <parameter name="Raw data file names"><file>\/Work\/III_mzML\/""" +file_name + """<\/file><\/parameter>/\' /Work/MZmine_FeatureFinder-batch.xml"""`

Client.execute(myinstance,command_list_1, options=['--writable-tmpfs'],quiet=False)`

Error message:
/bin/sed: -e expression #1, char 1: unknown command: `''

[vsoch]

Try putting the sed command in a bash script and running that. We split the woman’s with shlex and your sed FU probably doesn’t work well with that approach.

[jjacobson95]

Just wrote a quick python script to do it instead. Still would be a great thing to fix for users. I’m converting a docker pipeline to singularity for HPC and took quite a while to track down exactly where it was failing.

[vsoch]

Try running the command with subprocess as spython does and you will likely run into the same issue.

[vsoch]

spython isn’t a workflow tool - it’s a thin wrapper to running singularity. Likely a workflow tool would take a command and instead write to a temporary file first, but I consider that out of scope here because it’s not up to spython to make that decision for the user.

[jjacobson95]

Okay, your call of course! Just was surprised that the wrapper has a bit less functionality than the command line interface because it seems pretty well supported. Could be helpful to note in documentation for future users.

[vsoch]

The wrapper uses subprocess, and any tool that does similar will have similar issue with things like pipe or sed. If you’d like to do a PR to update docs with a suggestion I’d be happy to review that. I do not see singularity Python as being responsible for managing a workflow for which it’s really better practice to write complex commands to a bash script so they are properly run with a shell.