Fair use of the API?
Closed this issue · 33 comments
Hi!
This is a great project! ❤️
I would like to write and publish a PowerToys Run plugin that use your API.
Can I POST to https://dencode.com/dencode from the plugin?
Would that be fair use of the API?
Would you prefer that I set a specific user agent for those requests?
Thanks!
/ Henrik
Hi Henrik,
Thank you for asking.
I am glad that you are using DenCode.
Unfortunately, I would like you to refrain from calling the API endpoint from other system.
The README describes how to deploy to Google Cloud, so could you build your own server?
If you only want to use the endpoint, you can disable the UI by comment out like this:
/src/main/java/com/dencode/web/servlet/pages/IndexServlet.java
//@WebServlet("")
public class IndexServlet extends AbstractDencodeHttpServlet {If you have any other questions, feel free to ask.
Thank you!
Thanks for your reply.
Do you know of any public API that does what DenCode do?
I made DenCode because it doesn't exist in the world ;)
/dencode endpoint is designed specifically for the web UI, so it is not general purpose and its specifications are not stable.
If you don't mind, try using /dencode endpoint on a trial basis.
Please add a header like this: "X-Application-Id: com.laueriksson.henrik.powertoysrun-xxxx-plugin", if you call it. (Please name the ID appropriately and let me know later.)
I may ask you to stop if the server load increases, but let's try!
You can use the following icon for the search result.
https://github.com/mozq/dencode-web/tree/master/res/favicon
Maybe the plugin that copies a result to the clipboard?
I don't know what kind of specifications the new plug-in has, but it would be great if you could also display the link to DenCode site as shown below.
[method-name] value
e.g.
test-value
-> link to https://dencode.com/?v=test-value
string test-value
-> link to https://dencode.com/string?v=test-value
string/base64 test-value
base64 test-value
-> link to https://dencode.com/string/base64?v=test-value
hex test-value
-> link to https://dencode.com/string/hex?v=test-value
-> link to https://dencode.com/number/hex?v=test-value
Yes, exactly! I was not able to find any other service that is as good as DenCode.
I need a couple of weeks to finish other stuff, before I can start on a encode/decode plugin for PowerToys Run.
I will try out your API and be nice about it.
But I have a feeling that depending on libs, rater than an API, would be better for the plugin I have in mind.
I'll keep you posted in this issue.
Of course, I'm in favor of using local libraries.
It will not be affected by network issues, and the performance will be good.
The main reason DenCode is encoding on the server is because JavaScript is not good at character encoding conversion.
There are no restrictions in C#.
I'm a Mac user but will contribute if possible.
I look forward to the plugin being released!
Hi @mozq
Now I got time to start trying out your API.
Can I use this header:
X-Application-Id: Community.PowerToys.Run.Plugin.DenCode
Hi @hlaueriksson
Sure. Please let me know if you have any questions.
I have created a proof of concept of a DenCode PowerToys Run plugin:
Have you noticed my requests to the API?
Did you set the cookie header and get an error? I think you have already dealt with it. Cookie header are unnecessary and will be removed.
No other major errors were logged, so please go ahead.
I can see your screen record now. Great!!
If the input value and the result value are the same, it may be easier to find the desired result if you put them later in the display order or hide them.
Anyway, it seems to be easy to use, and smart behavior.
Did you set the cookie header and get an error? I think you have already dealt with it. Cookie header are unnecessary and will be removed.
No, I did not set any cookies.
If the input value and the result value are the same, it may be easier to find the desired result if you put them later in the display order or hide them.
Yes, that's smart. I will try this approach.
I'm parsing the config.properties and messages_en.properties files, to generate data for the plugin to use.
I found two issues:
hash.sha128does not have any translations inmessages_en.propertieshash.crc32.encHashCRC32should probably belabel.encHashCRC32
I have a question.
When the web is sending a POST request with method all.all, the payload looks like this:
{
"type": "all",
"method": "all.all",
"value": "Hello, world!",
"oe": "UTF-8",
"nl": "crlf",
"tz": "Africa/Algiers",
"options": {
"decStrMorseCodeVariant": "international",
"decCipherCaesarShift": "-3",
"decCipherEnigmaMachine": "I",
"decCipherEnigmaReflector": "UKW-A",
"decCipherEnigmaRotor4": "Beta",
"decCipherEnigmaRotor3": "I",
"decCipherEnigmaRotor2": "II",
"decCipherEnigmaRotor1": "III",
"decCipherEnigmaReflectorRing": "1",
"decCipherEnigmaRotor4Ring": "1",
"decCipherEnigmaRotor3Ring": "1",
"decCipherEnigmaRotor2Ring": "1",
"decCipherEnigmaRotor1Ring": "1",
"decCipherEnigmaReflectorPosition": "1",
"decCipherEnigmaRotor4Position": "1",
"decCipherEnigmaRotor3Position": "1",
"decCipherEnigmaRotor2Position": "1",
"decCipherEnigmaRotor1Position": "1",
"decCipherEnigmaPlugboard": "",
"decCipherEnigmaUhr": "0",
"decCipherEnigmaUkwd": "",
"decCipherScytaleKey": "2",
"decCipherRailFenceKey": "2",
"encStrBinSeparatorEach": "",
"encStrHexSeparatorEach": "",
"encStrHexCase": "lower",
"encStrURLEncodingSpace": "",
"encStrBase64LineBreakEach": "",
"encStrAscii85Variant": "z85",
"encStrUnicodeEscapeSurrogatePairFormat": "",
"encStrProgramStringQuotes": "double",
"encStrMorseCodeVariant": "international",
"encStrLineSortOrder": "asc",
"encNumEnShortScaleFractionalPartNotation": "",
"encNumEnShortScaleSystem": "",
"encDateISO8601DecimalMark": ".",
"encDateISO8601ExtDecimalMark": ".",
"encDateISO8601WeekDecimalMark": ".",
"encDateISO8601OrdinalDecimalMark": ".",
"encColorRGBFnNotation": "percentage",
"encCipherCaesarShift": "-3",
"encCipherEnigmaMachine": "I",
"encCipherEnigmaReflector": "UKW-A",
"encCipherEnigmaRotor4": "Beta",
"encCipherEnigmaRotor3": "I",
"encCipherEnigmaRotor2": "II",
"encCipherEnigmaRotor1": "III",
"encCipherEnigmaReflectorRing": "1",
"encCipherEnigmaRotor4Ring": "1",
"encCipherEnigmaRotor3Ring": "1",
"encCipherEnigmaRotor2Ring": "1",
"encCipherEnigmaRotor1Ring": "1",
"encCipherEnigmaReflectorPosition": "1",
"encCipherEnigmaRotor4Position": "1",
"encCipherEnigmaRotor3Position": "1",
"encCipherEnigmaRotor2Position": "1",
"encCipherEnigmaRotor1Position": "1",
"encCipherEnigmaPlugboard": "",
"encCipherEnigmaUhr": "0",
"encCipherEnigmaUkwd": "",
"encCipherJisKeyboardMode": "lenient",
"encCipherScytaleKey": "2",
"encCipherRailFenceKey": "2"
}
}How does the web know the default options? I don't want to hard code this in the plugin.
If I send a POST request myself with null options, the response is different than with the default options from the web request.
The issues with the .properties file have been fixed. Thank you for letting me know.
I've changed the default option values on the server side, could you try it with null?
options parameter can be empty.
"options": {}Thanks for fixing the .properties and the default options.
I noticed one diff when sending default options from the web vs empty options:
I also changed the following default values in web. Please change these options.
Sorry, I forgot to tell you that point.
(DenCode saves option value to browser's localStorage. Please clear it if you want to check the default again.)
"decCipherEnigmaRotor3": "III", // "I" -> "III"
"decCipherEnigmaRotor2": "II",
"decCipherEnigmaRotor1": "I", // "III" -> "I"
"encCipherEnigmaRotor3": "III", // "I" -> "III"
"encCipherEnigmaRotor2": "II",
"encCipherEnigmaRotor1": "I", // "III" -> "I"Yes, after clearing the local storage the response is identical. Thanks!
Hi @mozq and @hlaueriksson
I'm incredibly impressed with this project! The API usage thread has been super helpful, and I'm eager to start using it for decoding tasks like Quoted-printable code.
Could you please clarify a few details regarding API calls? I have some specific questions about encoding, body construction, and endpoints.
Here's a sample code I'd like to decode:
=FA=E5=E3=E4
Questions:
Encoding and Character Set:
Should I explicitly specify the encoding and character set when making a POST request?
If so, should I use a query string or include it in the request body?
Or does the API automatically detect these?
Request Body Construction:
What's the correct way to format the request body for decoding tasks?
Are there any specific examples or guidelines you can share?
Endpoint:
Which specific endpoint should I use for decoding?
Any additional information or documentation you can provide on API usage would be fantastic! I'm really keen to start exploring this tool's capabilities.
Thanks in advance for your help!
Truly appreciate your guidance.
Ram
Hi @TheRamSan
Thank you for your interest in this project.
This project aims to perform encoding/decoding via a web UI. Therefore, the API endpoint is designed for the UI and is not intended to be called from outside the UI.
I’m not sure your usage purpose or environment, but since Quoted-printable has a simple specification, so I recommend implementing decoding logic within your application.
Please refer to this code for decoding implementation.
Hello @mozq,
I appreciate your code and your fantastic website. The work you've done is impressive, and I'm truly grateful.
Currently, I'm using make.com to extract text from an eml file. Some of the text is encoded in Quoted-printable with the Windows-1255 character set, and I'm seeking a solution to decode it.
Unfortunately, within make.com, I'm restricted to running only vanilla JS code, and even that is not straightforward. Typically, if the required functionality is not available in make.com tools, we resort to making an HTTP request to an API that can provide the necessary information.
In this instance, I'm on the lookout for an API that can take a string as input and return the decoded text, preferably without the need to pre-select the character set, etc.
Do you happen to know of any such API?
Thanks again,
Ram
Happy New Year, @TheRamSan !
Windows-1255 has only few characters defined, so I mapped the codes to Unicode manually. (I'm Japanese and I usually use a crazy amount of characters...)
Please try the following vanilla JS code.
function decodeQP_windows1255(val) {
const WINDOWS1255_MAP = {
// See: https://en.wikipedia.org/wiki/Windows-1255
0x80: '€', 0x82: '‚', 0x83: 'ƒ', 0x84: '„', 0x85: '…', 0x86: '†', 0x87: '‡', 0x88: 'ˆ', 0x89: '‰', 0x8B: '‹',
0x91: '‘', 0x92: '’', 0x93: '“', 0x94: '”', 0x95: '•', 0x96: '–', 0x97: '—', 0x98: '˜', 0x99: '™', 0x9B: '›',
0xA0: '\{A0}', 0xA1: '¡', 0xA2: '¢', 0xA3: '£', 0xA4: '₪', 0xA5: '¥', 0xA6: '¦', 0xA7: '§', 0xA8: '¨', 0xA9: '©', 0xAA: '×', 0xAB: '«', 0xAC: '¬', 0xAD: '\{AD}', 0xAE: '®', 0xAF: '¯',
0xB0: '°', 0xB1: '±', 0xB2: '²', 0xB3: '³', 0xB4: '´', 0xB5: 'µ', 0xB6: '¶', 0xB7: '·', 0xB8: '¸', 0xB9: '¹', 0xBA: '÷', 0xBB: '»', 0xBC: '¼', 0xBD: '½', 0xBE: '¾', 0xBF: '¿',
0xC0: '\u{5B0}', 0xC1: '\u{5B1}', 0xC2: '\u{5B2}', 0xC3: '\u{5B3}', 0xC4: '\u{5B4}', 0xC5: '\u{5B5}', 0xC6: '\u{5B6}', 0xC7: '\u{5B7}', 0xC8: '\u{5B8}', 0xC9: '\u{5B9}', 0xCA: '\u{5BA}', 0xCB: '\u{5BB}', 0xCC: '\u{5BC}', 0xCD: '\u{5BD}', 0xCE: '\u{5BE}', 0xCF: '\u{5BF}',
0xD0: '\u{5C0}', 0xD1: '\u{5C1}', 0xD2: '\u{5C2}', 0xD3: '\u{5C3}', 0xD4: '\u{5F0}', 0xD5: '\u{5F1}', 0xD6: '\u{5F2}', 0xD7: '\u{5F3}', 0xD8: '\u{5F4}',
0xE0: '\u{5D0}', 0xE1: '\u{5D1}', 0xE2: '\u{5D2}', 0xE3: '\u{5D3}', 0xE4: '\u{5D4}', 0xE5: '\u{5D5}', 0xE6: '\u{5D6}', 0xE7: '\u{5D7}', 0xE8: '\u{5D8}', 0xE9: '\u{5D9}', 0xEA: '\u{5DA}', 0xEB: '\u{5DB}', 0xEC: '\u{5DC}', 0xED: '\u{5DD}', 0xEE: '\u{5DE}', 0xEF: '\u{5DF}',
0xF0: '\u{5E0}', 0xF1: '\u{5E1}', 0xF2: '\u{5E2}', 0xF3: '\u{5E3}', 0xF4: '\u{5E4}', 0xF5: '\u{5E5}', 0xF6: '\u{5E6}', 0xF7: '\u{5E7}', 0xF8: '\u{5E8}', 0xF9: '\u{5E9}', 0xFA: '\u{5EA}', 0xFD: '\u{200E}', 0xFE: '\u{200F}'
};
let s = '';
for (let i = 0; i < val.length; i++) {
const ch = val.charAt(i);
if (ch == '=') {
const nch = val.substring(i + 1, i + 3);
i += 2;
if (nch === '\r\n') {
// Soft line break
continue;
}
const cp = parseInt(nch, 16);
if (cp <= 0x7F) {
s += String.fromCharCode(cp);
} else {
s += WINDOWS1255_MAP[cp];
}
} else {
s += ch;
}
}
return s;
}
Hi @mozq
あけましておめでとうございます!
It's been a long time since I was in japan - but I do remember the nice decorations for the new year in oodori park in sapporo.
Thanks for the code, I will give it a got.
I really appreciate your help.
@TheRamSan Wow, ありがとうございます！どういたしまして！
I forgot 'u' in the mapping of 0xA0 and 0xAD. Please replace '\{}' to '\u{}'.
0xA0: '\u{A0}'
0xAD: '\u{AD}'
If you have any questions, please let me know.
P.S.
It is more better to use an Array instead of an Object.
const WINDOWS1255_MAP = [
'€', '', '‚', 'ƒ', ......
];
s += WINDOWS1255_MAP[cp - 0x80];
const WINDOWS1255_MAP = [
'€', '', '‚', 'ƒ', ......
];s += WINDOWS1255_MAP[cp - 0x80];
Thank you again for your help.
I really appreciate it.
Ram
Hi again @mozq
I finally got around to release the DenCode plugin for PowerToys Run:
The X-Application-Id header is Community.PowerToys.Run.Plugin.DenCode:
Hi @hlaueriksson
It is very easy to use and I love it!
Please let me know If you need an API that returns a list of encoding methods and names, etc.
I've added a link to your project page in DenCode's README.
Thank you.
I'm glad that you like the plugin. Thank you for an awesome site!
The dark mode support was a nice surprise when I first saw it 😎
I got the encoding methods from the old config.properties file:
If I parse all the .java files in the dencoder folder in the master branch, can I get the same data?
I have a test that generates the encoding method data for the plugin. It's included in the binaries.
I don't think you should add a API for this, if you don't plan to use it yourself.
I'm also glad you like dark mode😀
If I parse all the .java files in the dencoder folder in the master branch, can I get the same data?
Yes, you can parse the definitions from the .java files. However, definitions such as “useOe” are used for web UI, so you do not need to get them.
It's easier to parse “*.method=” in messages.properties. The file also includes names and descriptions.
Please note that the key such as “label.encStrBin=” will be changed to “string.bin.encStrBin=”, but I'm too lazy to do it yet :P Once the key changes, it becomes more easier to parse.
I’ve changed key name of dencoder functions in messages.properties as below.
string.bin.method=Bin String
string.bin.title=……
string.bin.desc=……
string.bin.tooltip=……
string.bin.func.encStrBin=Bin String
string.bin.func.decStrBin=Bin String
If dencoder methods or functions are added, renamed, or deleted in the future, please parse this file and synchronize it.
Thanks! I will take a look at messages.properties and update my parsing code.
Hi @hlaueriksson
FYI, here is a sample code for messages.properties parser.
(This code does not support "string.naming-convention" method. Please ignore the method.)
using System.Text.Json;
using System.Text.RegularExpressions;
Regex keyRegex = new("^(?<method>[^\\.]+\\.[^\\.]+)\\.(?<subkey>method|title|desc|toltip|func)(?:\\.(?<function>.+))?");
Dictionary<string, Dictionary<string, string>> methods = [];
Dictionary<string, Dictionary<string, string>> methodFunctions = [];
string messages = await new HttpClient().GetStringAsync("https://raw.githubusercontent.com/mozq/dencode-web/master/src/main/resources/messages.properties");
StringReader reader = new(messages);
string? line;
while ((line = reader.ReadLine()) != null)
{
int idx = line.IndexOf('=');
if (idx == -1)
{
continue;
}
string key = line[..idx];
string label = line[(idx + 1)..].Replace("\\\\", "\\");
Match m = keyRegex.Match(key);
if (!m.Success)
{
continue;
}
string method = m.Groups["method"].Value;
string subkey = m.Groups["subkey"].Value;
string function = m.Groups["function"].Value;
if (subkey == "method")
{
methods[method] = [];
methods[method]["key"] = method;
methods[method][subkey] = label;
methodFunctions[method] = [];
}
else if (subkey == "func")
{
methodFunctions[method][function] = label;
//methodFunctions[method[..method.IndexOf('.')] + ".all"][function] = label;
//methodFunctions["all.all"][function] = label;
}
else
{
methods[method][subkey] = label;
}
}
Console.WriteLine(JsonSerializer.Serialize(methods));
Console.WriteLine(JsonSerializer.Serialize(methodFunctions));Thanks for the sample code @mozq
I continued with my naive approach without regex:
Thank you for implementing it.