docutils-users Mailing List for Docutils: Documentation Utilities

Hei everyone,

the plan changed

0.22rc2 is for thursday may, 22

0.22 if nothing crops up tuesday june, 10th

all the best
 e

hei everyone

please try and test.

Many changes for details see https://docutils.sourceforge.io/HISTORY.html


reStructuredText:
  - Support `CSS3 units`_. This adds "ch", "rem", "vw", "vh", "vmin",
    "vmax", and "Q" to the `supported length units`__. Note that some
    output formats don't support all units.

  - New option "figname" for the `"figure"`_ directive.

  .. _CSS3 units: https://www.w3.org/TR/css-values-3/#lengths
  __ docs/ref/rst/restructuredtext.html#length-units

Document Tree / Docutils DTD
  - Allow multiple <term> elements in a `\<definition_list_item>`__
    (third-party writers may need adaption).
  - The first element in a <figure> may also be a <reference>
    (with nested "clickable" <image>).

  __ docs/ref/doctree.html#definition-list-item

Configuration changes
  - Make MathML the default math_output_ for the "html5" writer.

  - Change the default input_encoding_ from ``None`` (auto-detect) to
"utf-8".

  - Drop short options ``-i`` and ``-o``.
    Use the long equivalents ``--input-encoding`` and ``--output-encoding``.
    (See `command line interface`_ for the rationale.)

  - Rename configuration setting "output" to "output_path_".

  - The manpage writer now recognizes the sections [writers] and
    [manpage writer] with the new setting `text_references`_.

Output changes
  LaTeX:
     Don't wrap references with custom reference-label_ in a ``\hyperref``
     command. The "hyperref" package generates hyperlinks for labels by
     default, so there is no change in the PDF (except for "ref*").

     Stop requiring "ifthen.sty". Replace use of
     ``\ifthenelse{\isundefined...`` with the eTeX primitive ``\ifdefined``.

  HTML5:
     Unitless image_ size measures__ are written as <img> "width" and
     "hight" values instead of "style" rules.  The current behaviour
     is kept for values with units, so users may specify, e.g. ``:width:
     50px`` instead of ``:width: 50`` to override CSS stylesheet rules.

     __ docs/ref/doctree.html#measure

  manpage:
     Don't UPPERCASE section headings.
     Handle hyperlink references (see text_references_).

  null:
     The "null" writer output changed from None to the empty string.

     `publish_string()` now returns a `bytes` or `str` instance
     for all writers (as documented).

New objects
  `parsers.docutils_xml`
     parser for `Docutils XML`_ (e.g., the output of the "xml" writer).
     Provisional.

     Try ``docutils --parser=xml test/data/multiple-term-definitions.xml``
     or use the :parser: option of the `"include"`_ directive to include
     an XML file in a rST document.

  `nodes.Element.validate()`
     Raise `nodes.ValidationError` if the element does not comply with
     the `Docutils Document Model`_.
     Provisional.

  `writers.DoctreeTranslator`
     Generic Docutils document tree translator base class with
     `uri2path()` auxiliary method.
     Provisional.

Removed objects
  `core.Publisher.setup_option_parser()`
     internal, obsolete,
  `frontend.ConfigParser.get_section()`
     obsoleted by the configparser's "Mapping Protocol Access",
  `frontend.OptionParser.set_defaults_from_dict()`
     obsolete,
  `nodes.Element.set_class()`
     obsolete, append to Element['classes'] directly,
  `parsers.rst.directives.tables.CSVTable.decode_from_csv()`
     not required with Python 3,
  `parsers.rst.directives.tables.CSVTable.encode_from_csv()`
     not required with Python 3,
  `transforms.writer_aux.Compound`
     not used since Dec 2010,
  `utils.error_reporting`
     obsolete in Python 3,
  `utils.Reporter.set_conditions()`
     obsolete, set attributes via configuration settings or directly.

Removed localisations
  Mistranslations of the "admonition" directive name:
     Use "advies" (af), "varsel" (da), "warnhinweis" (de), "aviso" (es),
     "sciigo" (eo), "annonce" (fr), "avviso" (it), "advies" (nl),
     "zauważenie" (pl) (introduced in Docutils 0.21)
     or the English name "admonition".

New files
  ``docutils/parsers/rst/include/html-roles.txt``
     `Standard definition file`_ for additional roles matching HTML tags.

Removed files
  ``tools/rst2odt_prepstyles.py``
     Obsoleted by `writers.odf_odt.prepstyles`.
  ``docutils/utils/roman.py``
     Obsoleted by ``docutils/utils/_roman_numerals.py``

Bugfixes and improvements (see HISTORY_).

hello everyone,

the plan

* freeze now, only small corrections to the code till release
* 0.22rc1 on tuesday may 6th
* 0.22 on tuesday may 21th if no stopper shows up.

after more than a year a lot of changes have accumulated, some

* General
  - We have started to add type hints to Docutils (feature-request #87).
    This will be a complex programme of work and as such,
    for the time being, these type hints are "provisional"
    and should not be relied upon.

    By default, the Python interpreter treats type hints as annotations.
    Python >= 3.10 is required with active type hints
    (``typing.TYPE_CHECKING == True``).
* docutils/frontend.py
  - Drop short options ``-i`` and ``-o`` for ``--input-encoding``
    and ``--output-encoding``.
* very many internal changes, to clarify and stabilize
* docutils/writers/_html_base.py
  - Make MathML the default math_output_.
  - ...
* docutils/writers/latex2e
  - Support SVG image inclusion with the "svg" LaTeX package (see the
  - and more
* docutils/writers/manpage.py
  - better reference (URI) support

in detail https://docutils.sourceforge.io/HISTORY.html


all the best
 e

Dear Arne,

On 2025-03-25, Arne Skjærholt wrote:

...
> The use case I'm working with is making a solution for non-technical
> users to edit templates [...]
...

> Consider, the following fragment (obviously simplified, but it should
> communicate how I intend it to work):

> .. loop-construct:

>    This text is repeated, but with a varying value: :getfield:`fieldname`.

...
> What happens is that when a section is encountered that is a sibling
> of the current section level or higher in the section tree, the
> parsing state gets scrolled back to just before the offending section
> header, and EOFError is raised.[0] This error percolates up the stack,
> unwinding the section state as we go, until we get to a state where we
> can continue.


The Docutils document model allows section__ elements only inside the
main document or other sections.

__ https://docutils.sourceforge.io/docs/ref/doctree.html#section

There is no official support for section markup in nested content.

Did you nest section title syntax inside the directive content?

> The practical result of this is that the contents of my directive get
> truncated just before the section header each time around the loop,
> which is obviously not ideal. I've tried a couple of different
> approaches to getting this right, but nothing that quite works. The
> closest I've gotten involves looking at the return value from the
> nested parse, and when there's material left over taking the remaining
> lines, injecting the directive in front of those lines and indenting
> them before reinserting them into the input stream with
> self.state_machine.insert_input(). That sort of work, but I'm pretty
> sure it won't propagate the loop state correctly outside of the tiny
> example I've been using to debug this issue.

> Does anyone have suggestions on how to best handle this? I'm open to
> alternate ways of implementing the looping logic, but it really does
> need to loop I'm afraid and I think all my alternate ideas have so far
> run into some issue with threading the state correctly while still
> also getting everything into the output.

A typical approach would be to make the "loop-construct" directive insert
a <pending> element that stores the data during the parsing step
while a matching transform__ does the actual looping. 
See "class", "target-notes", "sectnum" and "contents" directives for
examples.

However, even then, starting sister-sections or parent-sections inside the
directive content would not be possible. 

Could you give a more detailled example how you plan to specify the
"underlying list" and the text to loop over?

Dear all,
I am working on an application of docutils/rST, but I'm running into a
snag with how sections find their place in the tree using exceptions
for control flow.

The use case I'm working with is making a solution for non-technical
users to edit templates that will be used to produce the final
documents. So basically a templating solution, but domain-specific and
couched in the terms of the domain rather than explicit logic. This is
probably somewhat counter to how rST is intended to be used, but I
still think it could be a good solution.

Mostly things are working out quite well, but the way sections are
implemented in the rST parsing of docutils interacts poorly with my
construct that has a looping logic. Consider, the following fragment
(obviously simplified, but it should communicate how I intend it to
work):

.. loop-construct:

   This text is repeated, but with a varying value: :getfield:`fieldname`.

The way it's implemented is that the loop-construct directive loops
over the underlying list, setting the value in a
contextvars.ContextVar and calling self.state.nested_parse on the
content each time, producing the elements that get inserted into the
output. This works quite well, until section headers get involved.
What happens is that when a section is encountered that is a sibling
of the current section level or higher in the section tree, the
parsing state gets scrolled back to just before the offending section
header, and EOFError is raised.[0] This error percolates up the stack,
unwinding the section state as we go, until we get to a state where we
can continue.

0: docutils.parsers.rst.states.py, line 361 in
RSTState.check_subsection in my copy of docutils-0.21.2

The practical result of this is that the contents of my directive get
truncated just before the section header each time around the loop,
which is obviously not ideal. I've tried a couple of different
approaches to getting this right, but nothing that quite works. The
closest I've gotten involves looking at the return value from the
nested parse, and when there's material left over taking the remaining
lines, injecting the directive in front of those lines and indenting
them before reinserting them into the input stream with
self.state_machine.insert_input(). That sort of work, but I'm pretty
sure it won't propagate the loop state correctly outside of the tiny
example I've been using to debug this issue.

Does anyone have suggestions on how to best handle this? I'm open to
alternate ways of implementing the looping logic, but it really does
need to loop I'm afraid and I think all my alternate ideas have so far
run into some issue with threading the state correctly while still
also getting everything into the output.

Arne
:wq

Hi Guenter, thanks for the information. Using the starting line of the next
sibling (or child) node sounds like a promising and elegant solution! I
usually walk the node tree section-by-section so that feels like it may
work very well. I'll try it out and follow up here with results.

On Mon, Jan 6, 2025 at 3:01 AM Guenter Milde via Docutils-users <
doc...@li...> wrote:

> On 2025-01-02, Kayce Basques via Docutils-users wrote:
> ...
> > Hello! I believe this is my first message in the Docutils community. I'm
> a
> > big fan of Sphinx and appreciate the core role that Docutils plays in
> > Sphinx. I subscribed to this list and am excited to be more active in the
> > community.
>
> Hello and welcome to the list.
>
> > I'm working on a Sphinx extension. In my doctree-resolved handler I
> > recursively walk through all section nodes. When the extension detects
> > something that can be improved in the underlying content, it's often
> > possible for the extension to make the edits automatically. Is the line
> > property of the Node class the only reference back to the underlying
> > reStructuredText?
>
> The internal attributes ``node.source`` and ``node.line`` hold the path
> or description of the input source and its start-line number respectively
> (cf. docutils.nodes.Node.source and docutils.nodes.Node.line).
>
> * ``node.source`` differs from the global document source for parts
>   included by the "include" directive.
> * Not every node has these attributes
>   (cf. https://sourceforge.net/p/docutils/feature-requests/41/).
>   For inline nodes, the attributes of the parent block-level node are used
>   in error reporting.
>
> Additionally, there is the internal `rawsource` attribute that is set in
> docutils.nodes.Element.__init__`. It comes with a caveat::
>
>         """The raw text from which this element was constructed.
>
>         For informative and debugging purposes. Don't rely on its value!
>
>         NOTE: some elements do not set this value (default '').
>         """
>
> > Just wanted to check that there's no explicit reference
> > to the end line of the node, and I'm expected to manually compute the end
> > line. The manual computation has been kinda error-prone and brittle for
> me
> > so far. Seems like the implementation could be much simpler and
> bulletproof
> > if reST explicitly gave me the end line. Just wanted to make sure there's
> > no better way to do this.
>
> For block-level elements, you may consider using the start line of the
> next node as a starting point.
>
> > One example of the manual computation I'm alluding to:
>
> > from docutils.nodes import section
>
> > def do_stuff(app, doc_tree, doc_name):
> >     for node in doc_tree.traverse(section):
> >         text = node.astext()
> >         start = node.line
> >         end = start + len(text.splitlines())  # Often incorrect
> >         …
> >         # A better approach might be to get the first and last lines
> >         # of text and search for those delimiters in the source
>
> Two suggestions (untested)::
>
> -         text = node.astext()
> +         text = node.rawsource or node.astext()
>
> or ::
>
> -         end = start + len(text.splitlines())  # Often incorrect
> +         end = node.next_node(descend=False, siblings=True,
> +                              ascend=True).line - 1
>
>
> I hope this may get you started on experimenting...
>
>
> A happy new year to all Docutils users and developers,
>
> Günter
>
>
>
> _______________________________________________
> Docutils-users mailing list
> Doc...@li...
> https://lists.sourceforge.net/lists/listinfo/docutils-users
>
> Please use "Reply All" to reply to the list.
>

On 2025-01-02, Kayce Basques via Docutils-users wrote:
...
> Hello! I believe this is my first message in the Docutils community. I'm a
> big fan of Sphinx and appreciate the core role that Docutils plays in
> Sphinx. I subscribed to this list and am excited to be more active in the
> community.

Hello and welcome to the list.

> I'm working on a Sphinx extension. In my doctree-resolved handler I
> recursively walk through all section nodes. When the extension detects
> something that can be improved in the underlying content, it's often
> possible for the extension to make the edits automatically. Is the line
> property of the Node class the only reference back to the underlying
> reStructuredText? 

The internal attributes ``node.source`` and ``node.line`` hold the path
or description of the input source and its start-line number respectively
(cf. docutils.nodes.Node.source and docutils.nodes.Node.line).

* ``node.source`` differs from the global document source for parts
  included by the "include" directive.
* Not every node has these attributes
  (cf. https://sourceforge.net/p/docutils/feature-requests/41/).
  For inline nodes, the attributes of the parent block-level node are used
  in error reporting.

Additionally, there is the internal `rawsource` attribute that is set in
docutils.nodes.Element.__init__`. It comes with a caveat::

        """The raw text from which this element was constructed.

        For informative and debugging purposes. Don't rely on its value!

        NOTE: some elements do not set this value (default '').
        """

> Just wanted to check that there's no explicit reference
> to the end line of the node, and I'm expected to manually compute the end
> line. The manual computation has been kinda error-prone and brittle for me
> so far. Seems like the implementation could be much simpler and bulletproof
> if reST explicitly gave me the end line. Just wanted to make sure there's
> no better way to do this.

For block-level elements, you may consider using the start line of the
next node as a starting point.

> One example of the manual computation I'm alluding to:

> from docutils.nodes import section

> def do_stuff(app, doc_tree, doc_name):
>     for node in doc_tree.traverse(section):
>         text = node.astext()
>         start = node.line
>         end = start + len(text.splitlines())  # Often incorrect
>         …
>         # A better approach might be to get the first and last lines
>         # of text and search for those delimiters in the source

Two suggestions (untested)::

-         text = node.astext()
+         text = node.rawsource or node.astext()

or ::

-         end = start + len(text.splitlines())  # Often incorrect
+         end = node.next_node(descend=False, siblings=True, 
+                              ascend=True).line - 1


I hope this may get you started on experimenting...


A happy new year to all Docutils users and developers,

Günter

Adam Turner sent me this response on the Write The Docs Slack:

"Docutils isn’t a format preserving parser — there have been various
experimental rST writers but nothing official yet. Your best bet is likely
using the source line information and reading from the source file
yourself."

https://writethedocs.slack.com/archives/C0899Q0G1/p1735852921645109

On Thu, Jan 2, 2025 at 2:58 PM Kayce Basques <ka...@go...> wrote:

> Hello! I believe this is my first message in the Docutils community. I'm a
> big fan of Sphinx and appreciate the core role that Docutils plays in
> Sphinx. I subscribed to this list and am excited to be more active in the
> community.
>
> I'm working on a Sphinx extension. In my doctree-resolved handler I
> recursively walk through all section nodes. When the extension detects
> something that can be improved in the underlying content, it's often
> possible for the extension to make the edits automatically. Is the line
> property of the Node class the only reference back to the underlying
> reStructuredText? Just wanted to check that there's no explicit reference
> to the end line of the node, and I'm expected to manually compute the end
> line. The manual computation has been kinda error-prone and brittle for me
> so far. Seems like the implementation could be much simpler and bulletproof
> if reST explicitly gave me the end line. Just wanted to make sure there's
> no better way to do this.
>
> One example of the manual computation I'm alluding to:
>
>
> from docutils.nodes import section
>
> def do_stuff(app, doc_tree, doc_name):
>     for node in doc_tree.traverse(section):
>         text = node.astext()
>         start = node.line
>         end = start + len(text.splitlines())  # Often incorrect
>         …
>         # A better approach might be to get the first and last lines
>         # of text and search for those delimiters in the source
>
> def setup(app):
>     app.connect('doctree-resolved', do_stuff)
>     return {
>         'version': '0.0.0',
>         'parallel_read_safe': True,
>         'parallel_write_safe': True,
>     }
>

Hello! I believe this is my first message in the Docutils community. I'm a
big fan of Sphinx and appreciate the core role that Docutils plays in
Sphinx. I subscribed to this list and am excited to be more active in the
community.

I'm working on a Sphinx extension. In my doctree-resolved handler I
recursively walk through all section nodes. When the extension detects
something that can be improved in the underlying content, it's often
possible for the extension to make the edits automatically. Is the line
property of the Node class the only reference back to the underlying
reStructuredText? Just wanted to check that there's no explicit reference
to the end line of the node, and I'm expected to manually compute the end
line. The manual computation has been kinda error-prone and brittle for me
so far. Seems like the implementation could be much simpler and bulletproof
if reST explicitly gave me the end line. Just wanted to make sure there's
no better way to do this.

One example of the manual computation I'm alluding to:


from docutils.nodes import section

def do_stuff(app, doc_tree, doc_name):
    for node in doc_tree.traverse(section):
        text = node.astext()
        start = node.line
        end = start + len(text.splitlines())  # Often incorrect
        …
        # A better approach might be to get the first and last lines
        # of text and search for those delimiters in the source

def setup(app):
    app.connect('doctree-resolved', do_stuff)
    return {
        'version': '0.0.0',
        'parallel_read_safe': True,
        'parallel_write_safe': True,
    }

Dear Wojciech,

On 2024-12-23, Wojciech Muła via Docutils-users wrote:

> I'm extending docutils to have shorter link. For example: :enwiki:`Docutils`
> expands into '<a href="https://en.wikipedia.org/Docutils>Docutils</a>'.
> I do that by using method `reference()`, like this:

> def enwiki_link_role(role, rawtext, text, lineno, inliner, options={}, content=[]):
>     set_classes(options)
>     ref, text = wikilink(text, 'en') # implementation detail
>     return [nodes.reference(rawtext, nodes.unescape(text), refuri=ref, **options)], []

> I want to add a title attribute, like '<a href=".." title="English
> Wikipedia on Docutils">...</a>'. Can't figure out how to do this, is it
> even possible?

In the `Docutils Document Format`__ (DocTree), the `"title" attribute`__
is only supported for the root element (<document>).

To get a HTML <a> element with a title, you would need to work around this
restriction:

* use "raw" HTML (https://docutils.sourceforge.io/docs/ref/doctree.html#raw)

* use a custom HTML writer that is able to write a "title" attribute for a
  <reference> DocTree element with either
  
  - a non-standard "title" attribute, or
  - a "special" class value (like "title-English-Wikipedia-on-Docutils")
  

Günter


__ https://docutils.sourceforge.io/docs/ref/docutils.dtd

https://docutils.sourceforge.io/docs/ref/doctree.html#title-1

Hi all!

I'm extending docutils to have shorter link. For example: :enwiki:`Docutils`
expands into '<a href="https://en.wikipedia.org/Docutils>Docutils</a>'.
I do that by using method `reference()`, like this:

def enwiki_link_role(role, rawtext, text, lineno, inliner, options={}, content=[]):
    set_classes(options)
    ref, text = wikilink(text, 'en') # implementation detail
    return [nodes.reference(rawtext, nodes.unescape(text), refuri=ref, **options)], []

I want to add a title attribute, like '<a href=".." title="English Wikipedia on Docutils">...</a>'.
Can't figure out how to do this, is it even possible?

Thanks!
I'm not a subscriber, please CC me.

regards
Wojciech

On 2024-10-14, Jonathan Gossage wrote:

> [-- Type: text/plain, Encoding:  --]

> Currently, the method of identifying a section using a title with
> underlines and optional overlines does most of what I need in docutils. It
> generates the following HTML:
> * HTML <section> start
> * HTML section title
> * HTML Id modifier to uniquely identify a section.
> * HTML </section> to mark the end of a section.

> What it does not do is allow me to specify an HTML class. What I want to do
> is to style all sections differently depending on the level - h1 to h6 of
> the section. I have done this manually by modifying the generated HTML and
> therefore have a working verification of the concept.

Why don't you style the sections based on the level tag? ::

    <style>
      h1 {color: green;}
      h2 {color: red;}
      h3 {background: blue;}
      ...
    </style>


> A simple way to achieve this seems to be to create a new directive. That
> allows me to add a class specification as an option to the directive but I
> am hoping that there may be a simpler solution. Any suggestions?

The ``.. class::`` directive can be used to pass a class value to a section,
e.g. with the rST snippet::
 
  some text
 
  .. class:: funny
  
  A section
  *********

rst2html5 will generate::

  <p>some text</p>
  <section class="funny" id="a-section">
  <h2>A section<a class="self-link" title="link to this section" href="#a-section"></a></h2>
  <p>yes.</p>
  </section>


> P.S. I am using Docutils from Sphinx using Sphinx 7.4.

In this case, you will have to use ``.. rst-class::`` instead of 
``.. class::``.

Cf. https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#directives


Günter

Currently, the method of identifying a section using a title with
underlines and optional overlines does most of what I need in docutils. It
generates the following HTML:
* HTML <section> start
* HTML section title
* HTML Id modifier to uniquely identify a section.
* HTML </section> to mark the end of a section.

What it does not do is allow me to specify an HTML class. What I want to do
is to style all sections differently depending on the level - h1 to h6 of
the section. I have done this manually by modifying the generated HTML and
therefore have a working verification of the concept.

A simple way to achieve this seems to be to create a new directive That
allows me to add a class specification as an option to the directive but I
am hoping that there may be a simpler solution. Any suggestions?

P.S. I am using Docutils from Sphinx using Sphinx 7.4.

-- 
Jonathan Gossage

Very tiny release

trove classifier supports languages Georgian and Catalan (Valencian)
so does docutils

and test failure fix.

one week or so waiting time for bug reports
then ... on to 0.22

all the best
 e

Release 0.21.2 consist of fix to tests
and support for languages Georgian and Catalan (Valencian)
both now in the pypi classifiers.

commit freeze till then and the week following for any problem reports

all the best
 e

Good evening

Adding a 0.21.post1 in release 0.21 did not work

This made a new release necessary.
It could have been 0.21.post2, but this might be another problem ...

This is the first time I tested --no-binary ... and it only worked from pypi
not from test.pypi, but it worked.

sorry for any inconvenience
 e

Hello to everyone,

Release 0.21 is out on pypi
  most of the work done by günter, kudos kudos kudos
  most of the fails on me

Release 0.21 (2024-04-09)
=========================

* General:

  - Drop support for Python 3.7 and 3.8.

  - Provide ``rst2*`` "console_scripts" `entry points`_
    (without the ``.py`` extension) instead of installing the
    ``rst2*.py`` `front end tools`_ in the binary PATH. [#]_

    Exceptions: ``rstpep2html.py`` and ``rst2odt_prepstyles.py``:

    - Use ``docutils --reader=pep --writer=pep_html`` for a PEP preview. [#]_
    - Use ``python -m docutils.writers.odf_odt.prepstyles``
      to `strip the page size`__ from an ODT writer stylesheet.

    __ docs/user/odt.html#page-size

  .. [#] Some Linux distributions already use the short names.
  .. [#] The final rendering is done by a Sphinx-based build system
         (cf. :PEP:`676`).

* reStructuredText:

  - Use the same CSV format for the ``:header:`` option and the main data
    of the "csv-table_" directive.

  - New option "loading" for the `"image" directive`_.
    Sets the new attribute loading__ of the <image> doctree element.

  __ docs/ref/doctree.html#loading

* Configuration changes:

  - New configuration setting root_prefix_.
    Configurable root directory for included files.

  - New configuration setting sources_ for the "buildhtml.py" application.

  - Simpler and more secure `input encoding`_ default behaviour:

    Do not use the locale encoding as fallback if Python is started in
    `UTF-8 mode`_. Stop using "latin1" as second fallback.

    Remove BOM (U+FEFF ZWNBSP at start of data) only if the `input_encoding`_
    configuration setting is None, '', 'utf-8-sig', 'utf-16', or 'utf-32'.
    Do not remove other ZWNBSPs.

* Output changes:

  HTML5:
    Stop setting the "footnote-reference" class value for footnote
    references. Use the CSS selector ``[role="doc-noteref"]``
    (works since Docutils 0.18, see minimal.css for examples).

    Fix MathML rendering problems in Chrome/Chromium based browsers.

    Embed SVG images as ``<svg>`` instead of data-URI.

  manpage:
    Use .EE/.EX macros for literal blocks.

    Render URI references (do not use .UR/.UE).

    Use box option for tables.

* Removed objects:

  `docutils.nodes.reprunicode`, `docutils.nodes.ensure_str()`
    Python 2 compatibility hacks
  `docutils.utils.Reporter.set_conditions()`
    obsolete
  `docutils.core.Publisher.setup_option_parser()`
    internal, obsolete

* New files:

  ``docutils/writers/html5_polyglot/italic-field-names.css``
    Alternative style for Docutils field-lists.

* Removed files:

  ``install.py``, ``setup.py``
    Metadata is now stored in ``pyproject.toml``,
    supported by pip_ since version 19.0 (2019-01-22).
    See README__ for installation alternatives.

  __ README.html#installation

* Bugfixes and improvements (see HISTORY_).

.. _input encoding: docs/api/publisher.html#encodings
.. _csv-table: docs/ref/rst/directives.html#csv-table
.. _"image" directive: docs/ref/rst/directives.html#image
.. _root_prefix: docs/user/config.html#root-prefix
.. _sources: docs/user/config.html#sources

Hi Anton,

thank you for the feedback.

On 2024-03-24, Anton Hvornum wrote:

> It appears there's some units missing from: 
> https://github.com/docutils/docutils/blob/b768e2626088711dec257b0847b563d02700a712/docutils/docutils/parsers/rst/directives/__init__.py#L240

> Namely: vh, vw, vmin, vmax, lvh, dvh, svw, lvw, dvw, svmin, lvmin, 
> dvmin, svmax, lvmax, dvmax, vi, svi, lvi, dvi, vb, svb, lvb and dvb

> These are part of the new HTML/CSS viewport units and are are/will be 
> part of CSS templates: 
> https://web.archive.org/web/20240114171725/https://www.terluinwebdesign.nl/en/css/incoming-20-new-css-viewport-units-svh-lvh-dvh-svw-lvw-dvw/

> If these are not compatible for some reason, perhaps they could be given 
> as a complementary argument to 
> https://github.com/docutils/docutils/blob/b768e2626088711dec257b0847b563d02700a712/docutils/docutils/parsers/rst/directives/__init__.py#L262C5-L262C23 
> for instance?

> I'm coming from a sphinx usage, where rendering HTML with CSS is the 
> base concept.

Did you see https://sourceforge.net/p/docutils/feature-requests/57/
"add vh and vw as allowable length units"?

In Docutils, we must take care to properly implement this for all supported
output formats (HTML, HTML4.1 + CSS1, LaTeX, ODT, manpages, XML)

This means we need to figure out what these units will be, e.g., on a
printed output or how to give adequate feedback if an output format does not
support the respective unit.

Suggestions welcome.

Günter

Hei everyone

0.21rc1 is out, a prerelease
to try out in an python environment install with the --pre argument

  python3 -m venv du3
  cd du3
  bin/activate
  pip install --pre docutils

0.21 final is due in two weeks.

Release 0.21.rc1 (2024-03-26)
=============================

* General:

  - Drop support for Python 3.7 and 3.8.

  - Provide ``rst2*`` "console_scripts" `entry points`_
    (without the ``.py`` extension) instead of installing the
    ``rst2*.py`` `front end tools`_ in the binary PATH. [#]_

    Exceptions: ``rstpep2html.py`` and ``rst2odt_prepstyles.py``:

    - Use ``docutils --reader=pep --writer=pep_html`` for a PEP preview. [#]_
    - Use ``python -m docutils.writers.odf_odt.prepstyles``
      to `strip the page size`__ from an ODT writer stylesheet.

    __ docs/user/odt.html#page-size

  .. [#] Some Linux distributions already use the short names.
  .. [#] The final rendering is done by a Sphinx-based build system
         (cf. :PEP:`676`).

* reStructuredText:

  - Use the same CSV format for the ``:header:`` option and the main data
    of the "csv-table_" directive.

  - New option "loading" for the `"image" directive`_.
    Sets the new attribute loading__ of the <image> doctree element.

  __ docs/ref/doctree.html#loading

* Configuration changes:

  - New configuration setting root_prefix_.
    Configurable root directory for included files.

  - New configuration setting sources_ for the "buildhtml.py" application.

  - Simpler and more secure `input encoding`_ default behaviour:

    Do not use the locale encoding as fallback if Python is started in
    `UTF-8 mode`_. Stop using "latin1" as second fallback.

    Remove BOM (U+FEFF ZWNBSP at start of data) only if the `input_encoding`_
    configuration setting is None, '', 'utf-8-sig', 'utf-16', or 'utf-32'.
    Do not remove other ZWNBSPs.

* Output changes:

  HTML5:
    Stop setting the "footnote-reference" class value for footnote
    references. Use the CSS selector ``[role="doc-noteref"]``
    (works since Docutils 0.18, see minimal.css for examples).

    Fix MathML rendering problems in Chrome/Chromium based browsers.

    Embed SVG images as ``<svg>`` instead of data-URI.

    Support video inclusion via ``<video>``.

  manpage:
    Use .EE/.EX macros for literal blocks.

    Render URI references (do not use .UR/.UE).

    Use box option for tables.

* Removed objects:

  `docutils.nodes.reprunicode`, `docutils.nodes.ensure_str()`
    Python 2 compatibility hacks
  `docutils.utils.Reporter.set_conditions()`
    obsolete
  `docutils.core.Publisher.setup_option_parser()`
    internal, obsolete

* New files:

  ``docutils/writers/html5_polyglot/italic-field-names.css``
    Alternative style for Docutils field-lists.

* Removed files:

  ``install.py``, ``setup.py``
    Metadata is now stored in ``pyproject.toml``,
    supported by pip_ since version 19.0 (2019-01-22).
    See README__ for installation alternatives.

  __ README.html#installation

* Bugfixes and improvements (see HISTORY_).

.. _input encoding: docs/api/publisher.html#encodings
.. _csv-table: docs/ref/rst/directives.html#csv-table
.. _"image" directive: docs/ref/rst/directives.html#image
.. _root_prefix: docs/user/config.html#root-prefix
.. _sources: docs/user/config.html#sources

For details see https://docutils.sourceforge.io/HISTORY.html

cheers
 e

Hei everyone,

Release 0.21 is planned for tuesday april 9 ... probably

all the best
 e

Change since 0.20
================

* General

  - Drop support for Python 3.7 and 3.8.
  - Updated build system to use Flit_ (patch #186 by Adam Turner).
    Removed ``setup.py``.
  - Provide ``rst2*`` "console_scripts" `entry points`_
    (without the ``.py`` extension) instead of installing the
    ``rst2*.py`` front end tools in the binary PATH.

  .. _Flit: https://github.com/pypa/flit/

* docs/ref/docutils.dtd

  - The <image> element accepts a new attribute "loading".

  - Fix definitions (no change to actual behaviour):

    * The <math_block> element uses the attribute "xml:space".
    * The <raw> element may contain text only (no inline elements).

* docutils/frontend.py

  - Allow `validate_*()` functions to be called with just the "value"
    argument but keep the legacy interface for use with optparse.
  - New function `frontend.validate_math_output()`.

* docutils/io.py

  - Simpler and more secure `input encoding`_ default behaviour:

    Do not use the locale encoding as fallback if Python is started in
    `UTF-8 mode`_. Stop using "latin1" as second fallback.

    Remove BOM (U+FEFF ZWNBSP at start of data) only if the "input_encoding"
    configuration setting is None, '', 'utf-8-sig', 'utf-16', or 'utf-32'.
    Do not remove other ZWNBSPs.

    .. _UTF-8 mode: https://docs.python.org/3/library/os.html#utf8-mode
    .. _input encoding: docs/api/publisher.html#encodings

  - Auto-close `FileInput.source` in case of reading/decoding errors.

* docutils/languages/, docutils/parsers/rst/languages/

  - Mark/Fix mistranslated localizations of the "admonition" directive
    name. In Docutils, "admonition" is used as a generic term for
    "advice"/"advisory"/"remark", not a reprimand.
  - Add support for Georgian language (patch #204 by Temuri Doghonadze).
  - Update/complete Catalan translations (patch #203 by Antoni Bella Pérez).

* docutils/nodes.py

  - Remove compatibility hacks `nodes.reprunicode` and `nodes.ensure_str()`.

* docutils/parsers/rst/directives/images.py

  - New "image" directive option "loading".

* docutils/parsers/rst/directives/tables.py

  - Use the same CSV format for the ``:header:`` option and the main data
    of the "csv-table" directive.

  - Move `parsers.rst.directives.Table.process_header_option()` method
    to `parsers.rst.directives.CSVTable`.

* docutils/parsers/rst/states.py

  - Don't split inside "< >" when parsing "option groups" (fixes bug #474).

* docutils/parsers/rst/directives/misc.py,
  docutils/parsers/rst/directives/tables.py

  - Consider the new root_prefix_ setting when including files with
    "include", "raw", or "csv-table" directives.

* docutils/utils/math/*

  - Use custom exception `utils.math.MathError` instead of
    abusing `SyntaxError` for LaTeX math syntax errors.
  - Unify interface of LaTeX -> MathML conversion functions.
    Improve error reporting.
  - Sort ℏ (`\hslash`) as "mathord", not "mathalpha".

* docutils/utils/math/latex2mathml.py

  - Generate "MathML Core" to fix rendering with Chromium/Android.

    Use CSS rules instead of the deprecated "columnalign" attribute
    to style <mtable> as "align" environment.

    Use Mathematical Alphanumeric Symbols instead of <mstyle> with
    "mathvariant" attribute.

  - Use <mi> instead of <mo> for character class "mathord".

  - Support "aligned" environment.

  - Eliminate side-effect on later import of "tex2unichar".

* docutils/utils/math/mathml_elements.py

  - New module defining MathML element classes
    (outsourced from latex2mathml.py).
  - Base MathML element classes on `xml.etree.ElementTree`.

* docutils/utils/roman.py

  - Update to version `1.4 <https://pypi.org/project/roman/4.1/>`__.
    Fixes feature-request #95 (license is now ZPL 2.1).

* docutils/utils/smartquotes.py

  - Pre-compile regexps once, not with every call of `educateQuotes()`
    (patch #206 by Chris Sewell).
  - Simplify regexps; skip replacement rules if there is nothing to replace.

* docutils/writers/html4css1/__init__.py

  - Support video inclusion via ``<object>`` tags.

* docutils/writers/html5_polyglot/\*.css

  - Move MathML styles to "minimal.css" (required for correct rendering).
  - Highlight heading of target section also with explicit hyperlink target.

* docutils/writers/_html_base.py

  - Stop setting the "footnote-reference" class value for footnote references.
    Since 0.18, you can use the CSS selector ``[role="doc-noteref"]``.
  - Support reading/embedding images also with "file:" URI.
  - Warn, if image scaling fails because the image file cannot be read.
  - Support video inclusion via ``<video>`` tags
    (moved here from writers/html5_polyglot/__init__.py).
  - New auxiliary method `HTMLTranslator.uri2imagepath()` ensures the
    image file can also be read when CWD and output directory differ.
  - Consider the root_prefix_ setting when converting an image URI
    to a local filesystem path.
  - New `\<image>`_ attribute "loading" overrides image_loading_ setting.
  - Embed SVG images as ``<svg>`` instead of data-URI.
    Fixes feature-request #100.
  - Generate system messages for errors/warnings during the writing stage
    (image transformations, math content conversion, ...).
  - Close ``<dt>`` element in `depart_term()` to allow a
    "definition_list_item" with multiple "terms" (cf. feature-request #60).
  - Link to the document "#top" from the ToC heading
    (unless toc_backlinks_ is False).

* docutils/writers/latex2e/__init__.py

  - Fix placement of hyperlink target (label) for tables (bug #440).
  - More compact LaTeX source for option-lists and description-lists
    (no change in output).

* docutils/writers/manpage.py

  - Skip footer to avoid the link to document source in the manpage.
  - Add multiple definition list term support, see feature #60.
  - Use ``.UR`` and ``.UE`` macros for reference markup.
  - Add preprocessor hinting tbl first line, see bug #477.
  - Change tbl-Tables using box option, see bug #475.
  - Apply literal block patch #205. Use ``.EE`` and ``.EX`` macros.
    Thanks to G. Branden Robinson.

* docutils/writers/odf_odt/__init__.py

  - Use context manager for image reading operations.
    Catch `URLError` when `urllib.request.urlopen()` fails.

  - Convert image URI to path if accessing a local file. Fixes bug #153.

* docutils/writers/s5_html/__init__.py

  - Warn if the S5 writer cannot copy the theme files.
  - Programmatic customization of theme_url_ setting no longer
    overridden by the default for theme_.

* tools/buildhtml.py

  - New configuration setting `sources`_.
  - Match `prune`_ values with `fnmatch.fnmatch()`.

Hi.

It appears there's some units missing from: 
https://github.com/docutils/docutils/blob/b768e2626088711dec257b0847b563d02700a712/docutils/docutils/parsers/rst/directives/__init__.py#L240

Namely: vh, vw, vmin, vmax, lvh, dvh, svw, lvw, dvw, svmin, lvmin, 
dvmin, svmax, lvmax, dvmax, vi, svi, lvi, dvi, vb, svb, lvb and dvb

These are part of the new HTML/CSS viewport units and are are/will be 
part of CSS templates: 
https://web.archive.org/web/20240114171725/https://www.terluinwebdesign.nl/en/css/incoming-20-new-css-viewport-units-svh-lvh-dvh-svw-lvw-dvw/

If these are not compatible for some reason, perhaps they could be given 
as a complementary argument to 
https://github.com/docutils/docutils/blob/b768e2626088711dec257b0847b563d02700a712/docutils/docutils/parsers/rst/directives/__init__.py#L262C5-L262C23 
for instance?

I'm coming from a sphinx usage, where rendering HTML with CSS is the 
base concept.
Where I might have a reST similar to:

```
Some Header
===========


.. list-table::

    * - .. figure:: /_static/a_picture.webp
           :width: 25dvw

```

And when running `make build` I'll hit: 
https://github.com/docutils/docutils/blob/b768e2626088711dec257b0847b563d02700a712/docutils/docutils/parsers/rst/directives/__init__.py#L256-L258

Because `units` is equal to `length_units` which is missing the unit dvw.

//Anton

Dear Tobias,

On 2024-02-17, Tobias Deiminger wrote:

> a short question on inline internal targets. Consider this example:

>  A short generic target _`is` going to cause conflicts. A target with
>  same name but different meaning _`is` likely to be defined somewhere
>  else.

> To avoid such ambiguous targets, is it possible to assign custom target
> names to inline internal targets that differ from their inline text?

I don't know of any way to achieve the exact result except raw HTML::

  .. role:: raw-html (raw)
     :format: html
  
  This short generic target 
  :raw-html:`<span class="target" id="is-2">is</span>`
  hopefully not going to cause conflicts. 

However, in most situations, it would be OK to simply use a larger part of
the sentence as target::

  A _`longer generic target` is not going to cause conflicts.
  
put the target on the containing block element::
  
  .. _block target:

  A "block target" with an id that differs from the content.
  
  .. container::
     :name: container-target
     
     (Almost) all directives accept the "name" option to specify a target
     name.
     
Hope this helps,
     
Günter     

Hi,
                                                                    
a short question on inline internal targets. Consider this example:
   
 A short generic target  _`is` going to cause conflicts. A target with same name
 but different meaning _`is` likely to be defined somewhere else.

To avoid such ambiguous targets, is it possible to assign custom target names to
inline internal targets that differ from their inline text?

Tobias

PS: Thanks for making docutils!

It was like "refrigerator blindness": I saw only what
I expected to see. Thanks for your willingness to
be so explicit.
Alan


On 8/30/2023 5:43 AM, Guenter Milde via Docutils-users wrote:
> The linked documentation tells you:
> 
>    * To install a `development version`_*from source*:
>    
>      1. Open a shell
>    
>      2. Go to the directory containing the file ``setup.py``.
>    
>      3. Install the package with **one** of the following commands::
>    
>             pip install -e .  # editable install
>             pip install .     # regular install
>             python setup.py   # regular install with setuptools
> 
> and ``pip install --help`` explains what an "editable install" is:
> 
>    -e, --editable <path/url>   Install a project in editable mode (i.e.
>                                setuptools "develop mode") from a local project
>                                path or a VCS url.

Dear Alan,

On 2023-08-29, Alan G. Isaac wrote:
> On 8/29/2023 5:01 PM, Guenter Milde via Docutils-users wrote:
>> On 2023-08-29, Alan G. Isaac wrote:
>>> Latest docutils from Subversion repository (vis svn update).
>>> Installed with
>>> 	python312 setup.py install
>>> Subsequent message from pip:
>>> 	DEPRECATION: Loading egg at c:\program files\python312\lib\site-packages\docutils-0.21b0.dev0-py3.12.egg is deprecated. pip 23.3 will enforce this behaviour
>>> change. A possible replacement is to use pip for package installation.

>>> I searched but did not find an answer to the question:
>>> what does this mean, and how to address it?

> On 8/29/2023 5:01 PM, Guenter Milde via Docutils-users wrote:
>> Latest tips at
>> https://docutils.sourceforge.io/README.html#installation

> Sure, but to install the current code on Subversion,
> I cannot use pip.  Right??

Actually, no. The linked documentation tells you:

  * To install a `development version`_ *from source*:
  
    1. Open a shell
  
    2. Go to the directory containing the file ``setup.py``.
  
    3. Install the package with **one** of the following commands::
  
           pip install -e .  # editable install
           pip install .     # regular install
           python setup.py   # regular install with setuptools

and ``pip install --help`` explains what an "editable install" is:

  -e, --editable <path/url>   Install a project in editable mode (i.e.
                              setuptools "develop mode") from a local project
                              path or a VCS url.


> I suspect (??) this serves as an alert to the
> docutils project that it must change how it
> uses setuptools before pip 23.3 is released.

Thank you for the pointer. The Docutils project distributes "wheels"
instead of "eggs" since 2015¹, so we should be safe for now.

We actually plan to replace "setup.py" in one of the next releases
https://sourceforge.net/p/docutils/patches/186/

Thanks,
Günter

¹Hence my question where you got the egg from.