Change query input enconding

[relu91]

Encoding the query JSON with __!_ and __#_ is not valid for production. We should investigate how to correctly escape " and ' characters and if it is not possible to escape it would be probably better to just use an URL encoder (ex: https://meyerweb.com/eric/tools/dencoder/ ) as an encoding algorithm.

[relu91]

I saw PR #6 but I still wonder if we need all of this, have you tested different running conditions and understood why the " or ' characters are missing? I need a clear answer to this question before choosing to use one enc/dec algorithm.

[FerrariAndrea]

In order to understand why the " or the ' is missing, we need to test the DApp from a real call (not from a local run), so with an app order, and the execution of the DApp in the workerpoll.
Because this issue shows up if the DApp is called from a workerpool.
I will try to do some tests with a new DApp that will just check the args (using the app order and the double/single quotes).

[relu91]

I will try to do some tests with a new DApp that will just check the args (using the app order and the double/single quotes).

Yes exactly, maybe you can leverage on our logging system to know more.

[FerrariAndrea]

I'm doing some experiments, for now, I don't have good news.
Tthe Iexec application that I'm using, just take and print the args, passed by the workerpool.

Starting from the run on my local machine with that 3 args:

 "This is a 'string' with ' ( 3 single quotes)" "this is a \"string\" with \" (double quotes)" ` double " and ' single quote string ` 

The output on mu cmd is:

[Wed Aug 24 2022 11:09:48 GMT+0000 (Coordinated Universal Time)][APP.test_param_0]: This is a 'string' with ' ( 3 single quotes)
[Wed Aug 24 2022 11:09:48 GMT+0000 (Coordinated Universal Time)][APP.test_param_1]: this is a "string" with " (double quotes)
[Wed Aug 24 2022 11:09:48 GMT+0000 (Coordinated Universal Time)][APP.test_param_2]: `

From the online logger we can see that as args array:

[Wed Aug 24 2022 11:09:48 GMT+0000 (Coordinated Universal Time)][APP]: [\"/usr/local/bin/ts-node\",\"/app/app.ts\",\"This is a 'string' with ' ( 3 single quotes)\",\"this is a \\\"string\\\" with \\\" (double quotes)\",\"`\",\"double\",\" and ' single quote string `\"]

Conclusion the "`" can't be used as a delimiter (like I did in this example) for the args, the single and the double quotes are working correctly.

About the workerpool execution, I can't execute the DApp in a workepool from the app-order (yet), but I can run it in a workerpool using the run command. I found some problems here.

Using simpler args in order to understand the problem, in that case, we will pass 3 arguments that are strings with spaces, so we need to use double quotes as delimiters for the args.
The argument of the command that will contain the args, must be a json from the Iexec documentation:
--params <json> # specify the params of the request, this option is reserved to an advanced usage (usage: --params '{"iexec_args":"dostuff","iexec_input_files":["https://example.com/file.zip"]}')

So the correct command should be like:
iexec app run --watch --chain viviani --trust 0 --params '{"iexec_args":"\"This is a string\" \"This is another string\" \"This is another string\""}'
This not work:
Command "iexec app run" failed with ValidationError: '{iexec_args:"This is a string" "This is another string" "This is another string"}' is not a valid params, must be a json

We can see that the JSON string removes the first level of double quotes, so I tried with:
iexec app run --watch --chain viviani --trust 0 --params '{\"iexec_args\":\"\\\"This is a string\\\" \\\"This is another string\\\" \\\"This is another string\\\"\"}'
And the result is:
Command "iexec app run" failed with ValidationError: '{"iexec_args":"\"This is not a valid params, must be a json
We can see, the double quotes are correct, but for some reason, Iexec cant handle that JSON as a valid one.

[relu91]

Conclusion the "`" can't be used as a delimiter (like I did in this example) for the args, the single and the double quotes are working correctly.

Yes ` is a well know delimiter for bash and it is used to call subshell. Not sure if you use PowerShell for your experiments but I guess it is used there similarly.

Using simpler args in order to understand the problem, in that case, we will pass 3 arguments that are strings with spaces, so we need to use double quotes as delimiters for the args.
The argument of the command that will contain the args, must be a json from the Iexec documentation:
--params # specify the params of the request, this option is reserved to an advanced usage (usage: --params '{"iexec_args":"dostuff","iexec_input_files":["https://example.com/file.zip"]}')

So the correct command should be like:
iexec app run --watch --chain viviani --trust 0 --params '{"iexec_args":""This is a string" "This is another string" "This is another string""}'
This not work:
Command "iexec app run" failed with ValidationError: '{iexec_args:"This is a string" "This is another string" "This is another string"}' is not a valid params, must be a json

We can see that the JSON string removes the first level of double quotes, so I tried with:
iexec app run --watch --chain viviani --trust 0 --params '{"iexec_args":"\"This is a string\" \"This is another string\" \"This is another string\""}'
And the result is:
Command "iexec app run" failed with ValidationError: '{"iexec_args":""This is not a valid params, must be a json
We can see, the double quotes are correct, but for some reason, Iexec cant handle that JSON as a valid one

Ok, something is weird, but why don't you use --args?

--args # specify the arguments to pass to the app

I would try this:

iexec app run --watch --chain viviani --trust 0 --args '"test first string" "test second string" "test" '

[FerrariAndrea]

Yea, this worked, you can see here the transaction with that params: Iexec explorer deal

I tested that too:
iexec app run --watch --chain viviani --trust 0 --args '"test 01: \\\" TEST" "test 02 \" TEST" "test 03 \"T\" TEST"'
And it worked: Iexec explorer deal

Now we are pretty sure that, the problem is the --param, --args seems to work.
We need to ensure how the DApp is run in the DESMO-LD demo (is the contract that runs it, right?).

[relu91]

Good news, can you try passing the JSON serialization of the query in this testing app? Perhaps it would be nice to add a simple JSON.parse call to get more information about what is going on.

We need to ensure how the DApp is run in the DESMO-LD demo (is the contract that runs it, right?).

After that test, we can move on with DESMO-LD. It is the SDK that is calling the execution.

[relu91]

Check also this StackOverflow answer about how to escape ' in bash.

[FerrariAndrea]

Confirmed it work with --args:
iexec app run --watch --chain viviani --trust 0 --args '"test 01: '"'"' TEST" "test 02 \" TEST" "test 03 '"'"'T'"'"' TEST"'
Iexec explorer deal