Support for custom units

[euronion]

Does the package support custom units provided as [MyUnit] / are there plans to support it?

Parsing strings with [...] fails. I tried defining the unit/dimension in the ureg before, but that doesn't help either.

MWE

from ucumvert import PintUcumRegistry
ureg = PintUcumRegistry()

# Example 1: Maybe works out of the box?
ureg.from_ucum("1.[MyUnit]") # Doesn't work

# Example 2: Maybe define the unit and dimension first?
ureg.define('MyUnit = [MyDimension]')
ureg.from_ucum("1.[MyUnit]") # Still doesn't work

resulting error message:

---------------------------------------------------------------------------
UnexpectedCharacters                      Traceback (most recent call last)
Cell In[104], line 2
      1 ureg.define('MyUnit = [MyDimension]')
----> 2 ureg.from_ucum("1.[MyUnit]")

File ~/Documents/GitHub/better-technology-data/.venv/lib/python3.12/site-packages/ucumvert/ucum_pint.py:373, in PintUcumRegistry.from_ucum(self, ucum_code)
    365 def from_ucum(self, ucum_code):
    366     """Transform an ucum_code to a pint unit.
    367 
    368     Parameters
   (...)    371         Ucum code as string.
    372     """
--> 373     parsed_data = self._ucum_parser.parse(ucum_code)
    374     return self._from_ucum_transformer(parsed_data)

File ~/Documents/GitHub/better-technology-data/.venv/lib/python3.12/site-packages/lark/lark.py:655, in Lark.parse(self, text, start, on_error)
    637 def parse(self, text: str, start: Optional[str]=None, on_error: 'Optional[Callable[[UnexpectedInput], bool]]'=None) -> 'ParseTree':
    638     """Parse the given text, according to the options provided.
    639 
    640     Parameters:
   (...)    653 
    654     """
--> 655     return self.parser.parse(text, start=start, on_error=on_error)

File ~/Documents/GitHub/better-technology-data/.venv/lib/python3.12/site-packages/lark/parser_frontends.py:104, in ParsingFrontend.parse(self, text, start, on_error)
    102 kw = {} if on_error is None else {'on_error': on_error}
    103 stream = self._make_lexer_thread(text)
--> 104 return self.parser.parse(stream, chosen_start, **kw)

File ~/Documents/GitHub/better-technology-data/.venv/lib/python3.12/site-packages/lark/parsers/earley.py:280, in Parser.parse(self, lexer, start)
    277     else:
    278         columns[0].add(item)
--> 280 to_scan = self._parse(lexer, columns, to_scan, start_symbol)
    282 # If the parse was successful, the start
    283 # symbol should have been completed in the last step of the Earley cycle, and will be in
    284 # this column. Find the item for the start_symbol, which is the root of the SPPF tree.
    285 solutions = dedup_list(n.node for n in columns[-1] if n.is_complete and n.node is not None and n.s == start_symbol and n.start == 0)

File ~/Documents/GitHub/better-technology-data/.venv/lib/python3.12/site-packages/lark/parsers/xearley.py:152, in Parser._parse(self, stream, columns, to_scan, start_symbol)
    149 for token in stream:
    150     self.predict_and_complete(i, to_scan, columns, transitives)
--> 152     to_scan = scan(i, to_scan)
    154     if token == '\n':
    155         text_line += 1

File ~/Documents/GitHub/better-technology-data/.venv/lib/python3.12/site-packages/lark/parsers/xearley.py:125, in Parser._parse.<locals>.scan(i, to_scan)
    123 if not next_set and not delayed_matches and not next_to_scan:
    124     considered_rules = list(sorted(to_scan, key=lambda key: key.rule.origin.name))
--> 125     raise UnexpectedCharacters(stream, i, text_line, text_column, {item.expect.name for item in to_scan},
    126                                set(to_scan), state=frozenset(i.s for i in to_scan),
    127                                considered_rules=considered_rules
    128                                )
    130 return next_to_scan

UnexpectedCharacters: No terminal matches '[' in the current parser context, at line 1 col 3

1.[MyUnit]
  ^
Expected one of: 
	* ANNOTATION
	* UNIT_METRIC
	* UNIT_NON_METRIC
	* PREFIX_SHORT
	* LPAR
	* PREFIX_LONG
	* FACTOR

[dalito]

You are the first to ask something like this. Currently the parser is strictly UCUM conform. The UCUM standard also does not specify a way to add custom units (as far as I know). Therefore, I have not yet thought about extending the parser based on custom pint definitions.

In principle it should be quite doable from the code side. The difficulty is agreeing on a specification of an UCUM+ (which could maybe also drop obscure units or support unicode).

[euronion]

Thanks for being open to this idea!

I went with square brackets [...] in the example above, as this is how UCUM seems to treat arbitrary units, e.g.:

ureg.from_ucum("1.[smoot]") # works, converts to to "1 smoot"
ureg.from_ucum("1.smoot") # does not work

So using square brackets seemed like a logical extension for what I found on the UCUM standard here:

1. Most arbitrary units in UCUM are also enclosed in square brackets.
2. Units enclosed in square brackets are unit atoms, so the boundary is marked clearly.
3. Between square brackets, the specification defines which characters are allowed and which are not ("Within a matching pair of square brackets the full range of characters 33–126 can be used.") -> No obscure unicode support.
4. Definitely not curly brackets {...}, as the standard is very clear that anything in curly brackets should be ignored.

I've only started reading into UCUM today, so let me know if I am totally off.

What I would not want is for PintUcumRegistry to silently accept custom units.
I think they should only be accepted after they have been defined with the registry (like me # Example 2 in the original post).

About our use case:
We are building a package where we need a standard for specifying units. We're likely going to go with pint, but seeing UCUM and ucumvert sparked our interested. (Funnily enough, it is the same application and domain as this original issue was about).
The custom units will be currencies or weird domain-specific units.