docutils-users Mailing List for Docutils: Documentation Utilities

Matthew Gilbert wrote:
> All,
> 
> I recently submitted a patch to MoinMoin which allows pretty much full 
> rst support within MoinMoin (so you can use rst to specify Wiki style 
> links, etc.). So far my patch is being ignored. I was wondering if 
> anyone had any advice as to how to approach the MoinMoin developers to 
> try to get them interested in accepting the patch. I haven't contributed 
> much to open source projects and am just not sure what is best way to 
> get their attention.
> 
> Also, I was wondering if anyone was interested in using (testing) it. 
> Are there any MoinMoin users on the list? Thanks _matt

I use a MoinMoin wiki and I would definitely be interested in using rst 
as the markup language.  If making the patch is not difficult I might be 
able to convince the maintainers to apply it.  Could you send me/point 
me to the patch and some instructions?

Best

Ivan

David Goodger wrote:

> Felix Wiemann wrote:
>
>> * I'd suggest adding built-in substitution definitions for "|--|" to
>>   en-dash and "|---|" to em-dash.
>
> I don't know about inserting a set of predefined substitution
> definitions into the parser.  But we could certainly include a set of
> substitution files in Docutils.  Then the author could do:
>
>      .. include:: <dashes.txt>

I'm not sure if the benefit is big enough enough to justify the effort
of adding such a feature and maintaining a set of 'standard'
substitution files.

Probably it's best to just require the document author to include his
own substitution file(s).

>> IMO the trailing space should be made omittable.
>
> We'd still need a leading space.

I thought the leading space was optional; seems I got the current syntax
wrong...  8-)

> But this gave me an idea.  In conjunction with a change to the
> "unicode" directive, substitutions could become context-sensitive.  We
> could add a "trim" option to the "unicode" directive, as follows:
>
>      .. |--| unicode:: U+02013 .. EN DASH
>         :trim:
>      .. |---| unicode:: U+02014 .. EM DASH
>         :trim:

Looks nice.

But what about multi-line unicode definitions?  Recognize the option iff
the last line is ':trim:'?  Not too elegant but it could work.

> And other characters can be used as markup delimiters, not just
> spaces.  For example, hyphens can be used.

I think that would be over-engineering.  We don't *really* need it, do
we?

-- 
When replying to my email address, please ensure
that the mail header contains 'Felix Wiemann'.

http://www.ososo.de/

All,

I recently submitted a patch to MoinMoin which allows pretty much full 
rst support within MoinMoin (so you can use rst to specify Wiki style 
links, etc.). So far my patch is being ignored. I was wondering if 
anyone had any advice as to how to approach the MoinMoin developers to 
try to get them interested in accepting the patch. I haven't contributed 
much to open source projects and am just not sure what is best way to 
get their attention.

Also, I was wondering if anyone was interested in using (testing) it. 
Are there any MoinMoin users on the list? Thanks _matt

[Felix Wiemann]
 > With the current implementations, some documents are specifically
 > written for the LaTeX writer (because they rely on the
 > dash-transformation) and some are written specifically for the HTML
 > writer (because they rely on multiple dashes not to be transformed).

That's bad.

 > So we have a problem which needs to be solved.

Yes.  IMO, it's a bug that the LaTeX writer implicitly performs any
dash transformation at all.  It's a dangerous convenience.

 > A somewhat radical but nonetheless simple and effective solution
 > might be to deactivate the transformation in the LaTeX writer.

+1

 > However, then it should be possible to easily enter en-/em-dashes
 > with ASCII characters.
 >
 > * I'd suggest adding built-in substitution definitions for "|--|" to
 >   en-dash and "|---|" to em-dash.

I don't know about inserting a set of predefined substitution
definitions into the parser.  But we could certainly include a set of
substitution files in Docutils.  Then the author could do:

     .. include:: <dashes.txt>

See <http://docutils.sf.net/docs/dev/todo.html#misc.include>; more
below.

 > * And it would be necessary to write em-dashes without spaces around.

Are you saying that substitution references should not require any
delimiters?  That won't work.  Substitution references are like any
other reST inline markup; the start-string and end-string recognition
rules must apply in order to avoid ambiguity
(http://docutils.sf.net/docs/ref/rst/restructuredtext.html#inline-markup).

This is the best we can do right now:

$ quicktest.py
foo\ |---|\ bar
<document source="<stdin>">
     <paragraph>
         foo
         <substitution_reference refname="---">
             ---
         bar

 > IMO the trailing space should be made omittable.

We'd still need a leading space.  With an omissible trailing space,
the best we'd be able to do would be

     foo\ |---|bar

That isn't much better than the current "foo\ |---|\ bar".  Certainly
not worth the ambiguity and effort.

But this gave me an idea.  In conjunction with a change to the
"unicode" directive, substitutions could become context-sensitive.  We
could add a "trim" option to the "unicode" directive, as follows:

     .. |--| unicode:: U+02013 .. EN DASH
        :trim:
     .. |---| unicode:: U+02014 .. EM DASH
        :trim:

Then this input:

     foo |---| bar

could become this output:

     foo&mdash;bar

And other characters can be used as markup delimiters, not just
spaces.  For example, hyphens can be used.  Alternative substitution
definitions I'm thinking of include:

     .. |M| unicode:: U+02014 .. EM DASH
        :trim: -
     .. |N| unicode:: U+02013 .. EN DASH
        :trim: -
     .. |?| unicode:: U+000AD .. SOFT HYPHEN
        :trim: -
     .. |!| unicode:: U+02011 .. NON-BREAKING HYPHEN
        :trim: -
     .. |#| unicode:: U+02012 .. FIGURE DASH
        :trim: -

So an em-dash could be written like this, similar to the proofreaders'
mark:

     foo-|M|-bar

and would produce (the equivalent of) this:

     foo&mdash;bar

Alternatively, XML entity names (|mdash|) could be used instead of the
cryptic symbols above (|M|).

Many space characters could also be defined:

     .. |emsp| unicode:: U+02003 .. EM SPACE
        :trim:
     .. |ensp| unicode:: U+02002 .. EN SPACE
        :trim:
     .. |puncsp| unicode:: U+02008 .. PUNCTUATION SPACE
        :trim:
     .. |numsp| unicode:: U+02007 .. DIGIT SPACE
        :trim:
     .. |thinsp| unicode:: U+02009 .. THIN SPACE
        :trim:
     .. |hairsp| unicode:: U+0200A .. HAIR SPACE
        :trim:
     .. |0sp| unicode:: U+0200B .. ZERO WIDTH SPACE
        :trim:
     .. |zwnj| unicode:: U+0200C .. ZERO WIDTH NON-JOINER
        :trim:
     .. |zwj| unicode:: U+0200D .. ZERO WIDTH JOINER
        :trim:
     .. |nbsp| unicode:: U+000A0 .. NO-BREAK SPACE
        :trim:

In fact, all of the character entity files in the add-on package
(http://docutils.sourceforge.net/tmp/charents.tgz, which should come
standard with Docutils) could have space-trimmed alternatives.

Discussion welcome.

-- 
David Goodger <http://python.net/~goodger>

[Felix Wiemann]
 > David Goodger wrote:
 >> We must resist the temptation to guess.
 >
 > But if the user explicitly asks for guessing? ::

Sure, that would be OK.  Perhaps that ought to be the mechanism for
simple tables too.

 >> I don't see why we should leave out center-alignment.  Mark
 >> Nodine's original logic works.
 >
 > Not reliably.
 >
 > =========== ===========
 >  Head 1      Head 2
 > =========== ===========
 >  Row 1       some

I see.  I don't format my tables this way, but others may.  Perhaps
center-alignment-detection should be explicit.  Something like:

     .. table:: :auto-align: +center

 > And people might be tempted to get head-centering:
 >
 > ====================== ======================
 >  **Heading column 1**   **Heading column 2**
 > Row 1                  Row 1
 > Row 2                  Row 2
 > The headings aren't    Row 3
 > real headings but just Row 4
 > bold and centered      Row 6
 > normal cells.          Row 7
 > ====================== ======================

That's misuse/abuse, but we can't prevent that.

 > After all, I don't think the centering is needed very often, so it's
 > safer not to include it.

Perhaps safer, but less functional too.  People will want centering.

-- 
David Goodger <http://python.net/~goodger>

David Goodger wrote:

> We must resist the temptation to guess.

But if the user explicitly asks for guessing? ::

.. compound::

   ::

       .. table::
          :auto-align:

   or something like that (``:guess-alignment:``, pragma-directive,
   etc.).

>> The only thing I'd be confident about that it doesn't cause too much
>> trouble is your proposed one-line-in-simple-table alignment-guessing
>> algorithm, but only if we leave out 'centered' (because that
>> presumably isn't needed as often as 'right-aligned' and there might
>> be more false-positives).
>
> I don't see why we should leave out center-alignment.  Mark Nodine's
> original logic works.

Not reliably.

=========== ===========
 Head 1      Head 2
=========== ===========
 Row 1       some
 Row 2       weird
 Row 3       way
 Row 4       to
 Row 5       indent
 Row 6       things
=========== ===========

Looks pretty left-aligned, hum?  The algorithm would center it.

And people might be tempted to get head-centering:

====================== ======================
 **Heading column 1**   **Heading column 2**
Row 1                  Row 1
Row 2                  Row 2
The headings aren't    Row 3
real headings but just Row 4
bold and centered      Row 6
normal cells.          Row 7
====================== ======================

After all, I don't think the centering is needed very often, so it's
safer not to include it.

If more sophisticated aligning is needed, this simple cell-alignment
algorithm won't suffice anyway, with or without centering.

-- 
When replying to my email address, please ensure
that the mail header contains 'Felix Wiemann'.

http://www.ososo.de/

With the current implementations, some documents are specifically
written for the LaTeX writer (because they rely on the
dash-transformation) and some are written specifically for the HTML
writer (because they rely on multiple dashes not to be transformed).

Considering the commonness of both en-/em-dashes and unix-style options,
it is indeed probable that this writer dependence exists for many
documents.

Furthermore, in LaTeX there are probably frequent false-positives,
because the dash-transformation is applied unconditionally, and it isn't
even escapable.

So we have a problem which needs to be solved.

After re-reading David's posting, I too finally came to the conclusion
that an intelligent guessing-algorithm might be a bad idea.

A somewhat radical but nonetheless simple and effective solution might
be to deactivate the transformation in the LaTeX writer.

However, then it should be possible to easily enter en-/em-dashes with
ASCII characters.

* I'd suggest adding built-in substitution definitions for "|--|" to
  en-dash and "|---|" to em-dash.

* And it would be necessary to write em-dashes without spaces around.

The latter thing doesn't work, however:

$ quicktest.py 
foo|---|
<document source="<stdin>">
    <paragraph>
        foo|---|
$ quicktest.py 
foo|---|bar
<stdin>:1: (WARNING/2) Inline substitution_reference start-string without end-string.
<document source="<stdin>">
    <paragraph>
        foo|---
        <problematic id="id2" refid="id1">
            |
        bar
    <system_message backrefs="id2" id="id1" level="2" line="1" source="<stdin>" type="WARNING">
        <paragraph>
            Inline substitution_reference start-string without end-string.

IMO the trailing space should be made omittable.  (I think it won't
cause any existing documents to break, because this change would only
turn invalid constructs into valid ones.)

-- 
When replying to my email address, please ensure
that the mail header contains 'Felix Wiemann'.

http://www.ososo.de/

[Rune Froysa]
> The processing of rst files can be altered with a configuration-file
> and command-line options.  As far as I can see, none of these options
> support adding custom directives.

True.

> Thus one will have to make customzied versions of the rst2* scripts
> for this.
> 
> I would like to be able to somehow specify my custom directives, and
> perhaps doing other customizations, without modifying these scripts.
> There are several ways to allow this, for example:
> 
> - configuration-file parameter
> - command-line option
> - environment variable
> 
> Any of these options could for instance point to a python script or
> similar that was evaluated in the proper context.

Or a directory that contains Python modules that conform to a certain
interface, for true plug-ins.

> Has something like this been considered for inclusion in docutils (I
> notice that the todo list mentions "Allow directives to be added at
> run-time?")?

Yes, it's been talked about before.

> I could consider proposing a patch.

That would be great!

> Are there any preferences regarding how it is done?

It could be a combination of all 3 methods you mentioned.

I suggest that the first step would be to discuss it on the
docutils-develop list.  That's where the developers of Docutils hang
out.

-- 
David Goodger <http://python.net/~goodger>

The processing of rst files can be altered with a configuration-file
and command-line options.  As far as I can see, none of these options
support adding custom directives.  Thus one will have to make
customzied versions of the rst2* scripts for this.

I would like to be able to somehow specify my custom directives, and
perhaps doing other customizations, without modifying these scripts.
There are several ways to allow this, for example:

- configuration-file parameter
- command-line option
- environment variable

Any of these options could for instance point to a python script or
similar that was evaluated in the proper context.

Has something like this been considered for inclusion in docutils (I
notice that the todo list mentions "Allow directives to be added at
run-time?")?  I could consider proposing a patch.  Are there any
preferences regarding how it is done?

Regards,
-- 
Rune Frøysa

[Rune Froysa]
 > I certainly see the need for a switch to explicitly enable this
 > feature when working with non-trusted sources.

Perhaps a command-line switch would be protection enough.  Perhaps an
interactive query would be necessary.  I wouldn't be comfortable
relying on a config file entry for this though; to easy to forget that
it's there.

 > IMHO numerous sysinclude directives that calls scripts to extract
 > information from other documents (python/sql sources etc.; by
 > keeping the actual documentation in the sourcecode it is easier to
 > assert that it is up-to-date) is cleaner than numerous include
 > statements that referes to temporary files generated by a wrapper.

Cleaner, perhaps, but dangerous.

 > I do not agree that such a feature makes reST a programming
 > language,

A wrapper (vehicle, delivery system) for executable code, then.

 > nor does allowing environment-variables as part of the path to
 > included files (avoids forcing a spesific directory-structure when
 > dealing with documents from multiple CVS repositories/temporary
 > directories).

That moves reST toward being a templating system.  That's not reST's
job, and it would only add complexity without really solving the
problem properly.

There is a much safer and more robust way to accomplish this: use a
templating system (or build your own ;-).  See question 2.19 of the
FAQ: <http://docutils.sourceforge.net/FAQ.html>.

I'm not against anybody writing or using a "system" directive, but I
am against including it in core Docutils.  It would be a powerful and
potentially very dangerous feature.  Perhaps this wouldn't be such an
issue if there was a better "plug-in" system for 3rd-party directives,
so they can be discovered and added at run-time.  Discussion is
welcome.

 > I'm a bit surprised that the todo list contains entries for options
 > that will not be accepted into core,

Sometimes questionable ideas are proposed and added to the to-do list,
usually marked with question marks and with misgivings spelled out.
Sometimes an idea will linger in the to-do list for quite a while, for
various reasons: nobody's had time to implemented it, the issues are
complex, it takes a while for opinions to form, etc.  Once a final
decision is made, the to-do list entry may be moved over to the
docs/dev/rst/alternatives.txt file, and classified under "To Do",
"... Or Not To Do?", "Implemented", "Not Implemented", or "Tabled".
In this particular case, I think it was a matter of opinions (mine,
anyway) taking time to form.  Opinions can (and do) change though.

 > and would apreciate if the todo list could be updated to reflect
 > this.

There's only so much volunteer time to go around.  We could use more
of yours here!

-- 
David Goodger <http://python.net/~goodger>

David Goodger <go...@py...> writes:

> I'm not so keen on sysinclude.  Such an idea has been around for a
> long time (see http://docutils.sf.net/docs/dev/todo.html#misc-system),
> but never included in core Docutils because of its inherent danger and
> lack of portability between platforms.  ReStructuredText should not
> turn into a programming language, or a carrier of anything resembling
> executable code.  I don't want standard reST to ever even be able to
> be perverted into becoming a vector for malware.

I certainly see the need for a switch to explicitly enable this
feature when working with non-trusted sources.

IMHO numerous sysinclude directives that calls scripts to extract
information from other documents (python/sql sources etc.; by keeping
the actual documentation in the sourcecode it is easier to assert that
it is up-to-date) is cleaner than numerous include statements that
referes to temporary files generated by a wrapper.  I do not agree
that such a feature makes reST a programming language, nor does
allowing environment-variables as part of the path to included files
(avoids forcing a spesific directory-structure when dealing with
documents from multiple CVS repositories/temporary directories).

I'm a bit surprised that the todo list contains entries for options
that will not be accepted into core, and would apreciate if the todo
list could be updated to reflect this.

Regards, 
-- 
Rune Frøysa

[Rune Froysa]
 > I have written a patch that defines two new directives, namespace
 > and sysinclude.

I like the namespace directive; it could be very useful.  I need to
take a closer look at it though, to make sure there are no problems.
One problem I already see is that the namespace is joined to the ID
with a period (".").  That won't work with CSS (see the docstring for
docutils.nodes.make_id).  Perhaps double hyphens ("--", which make_id
never produces) instead?

I'm not so keen on sysinclude.  Such an idea has been around for a
long time (see http://docutils.sf.net/docs/dev/todo.html#misc-system),
but never included in core Docutils because of its inherent danger and
lack of portability between platforms.  ReStructuredText should not
turn into a programming language, or a carrier of anything resembling
executable code.  I don't want standard reST to ever even be able to
be perverted into becoming a vector for malware.

-- 
David Goodger <http://python.net/~goodger>

[Felix Wiemann]
 > David Goodger wrote:
 >> How is the text in this cell aligned?
 >>
 >> +------+
 >> | word |
 >> +------+
 >>
 >> It's not centered!
 >
 > It's left aligned.
 >
 > If you assume that people aren't indenting more than one space
 > (which I'm not sure of if it's correct),

I don't think that's safe to assume.

 > Maybe we need more intelligent center recognition.
 >
 > +--------------+
 > |  word        |
 > +--------------+
 >
 > Looks more left-aligned than centered.
 >
 > But I think it would be possible to get some nice guessing logic.

That's the point.  We must resist the temptation to guess.

 > =====  ===========================
 > row 1      But if we right-align
 >            the whole block and not
 >            every single line, it
 >            might work.

No, that's an indented left-aligned paragraph.

 > -----  ---------------------------
 > row 2     First paragraph.
 >
 >           Second paragraph.
 >
 >           Everything in this cell
 >           is right-aligned,
 >           because the whole cell
 >           block is right-aligned.

The whole cell block is indented, not right-aligned.

 >           This is still more
 >           right-aligned than
 >           centered (3:1), but it
 >           requires some logic to
 >           recognize.
 > =====  ===========================

That logic would be fragile and unreliable.

 > The only thing I'd be confident about that it doesn't cause too much
 > trouble is your proposed one-line-in-simple-table alignment-guessing
 > algorithm, but only if we leave out 'centered' (because that
 > presumably isn't needed as often as 'right-aligned' and there might
 > be more false-positives).

I don't see why we should leave out center-alignment.  Mark Nodine's
original logic works.

 > The other logic I described above could be applied when passing an
 > :auto-align: option to the table directive, which would ensure that
 > existing documents don't break.

Yes, that could work for grid tables.  Another possibility is a pragma
directive.  For simple tables, I think Mark's idea could be applied
without danger.

-- 
David Goodger <http://python.net/~goodger>

David Goodger wrote:

> Felix Wiemann wrote:
>
>> With some more logic, it would be possible for grid tables too,
>> wouldn't it?
>
> I don't think so.  Grid tables are ambiguous that way.  When
> writing/drawing a grid table, I invariably include a left & right
> margin in each table cell.  The spec specifically allows for such
> margins, and the parser handles them transparently.  How is the text
> in this cell aligned?
>
> +------+
> | word |
> +------+
>
> It's not centered!

It's left aligned.  If you assume that people aren't indenting more than
one space (which I'm not sure of if it's correct), you can have the
following logic:

If (left indent <= 1 char), then align left,
else if (right indent <= 1 char), then align right,
else center.

Maybe we need more intelligent center recognition.

+--------------+
|  word        |
+--------------+

Looks more left-aligned than centered.

But I think it would be possible to get some nice guessing logic.

>>> This wouldn't make sense for anything but single one-line
>>> paragraphs per cell.
>>
>> With "make sense", you mean "useful" or "syntactically possible"?
>
> "Possible".  Counter-examples:
>
> =====  =======================
> row 1   if we try to make this
>               right-aligned, it
>            ends up looking like
>                definition lists
>              or bad indentation

=====  ===========================
row 1      But if we right-align
           the whole block and not
           every single line, it
           might work.

> -----  -----------------------
> row 2     here's one paragraph
>
>                  here's another
>
>         the indentation above
>         implies block quotes,
>         not right alignment
> =====  =======================

-----  ---------------------------
row 2     First paragraph.

          Second paragraph.      

          Everything in this cell
          is right-aligned,
          because the whole cell
          block is right-aligned.

          In this block, the
          indentation on the left
          is 3 chars, but the
          indentation on the
          right is 1 char.

          This is still more
          right-aligned than
          centered (3:1), but it
          requires some logic to
          recognize.
=====  ===========================

The problem is that all that guessing logic would break existing
documents.

The only thing I'd be confident about that it doesn't cause too much
trouble is your proposed one-line-in-simple-table alignment-guessing
algorithm, but only if we leave out 'centered' (because that presumably
isn't needed as often as 'right-aligned' and there might be more
false-positives).  So the logic would be:

If a cell of a simple table contains 1 line of text and there is no
space on the right to the cell border and there is at least one char of
space on the left to the cell border, the cell is right-aligned,
otherwise it is left-aligned.

Example:

======= ======
left     right
 left      left
  left   left
  right left
======= ======

The other logic I described above could be applied when passing an
:auto-align: option to the table directive, which would ensure that
existing documents don't break.

I'm not familiar with the tableparser implementation, which would
probably be a requirement for implementing all this.  So I probably
can't do too much.  But if you write a function extracting the distance
to the left and to the right border of a cell, I could write a
guessing-function if you like (but that wouldn't be very difficult
anyway, I reckon).

-- 
When replying to my email address, please ensure
that the mail header contains 'Felix Wiemann'.

http://www.ososo.de/

[David Goodger]
 >> You may be opening up a big can of worms.  Once the underlying
 >> system is there, won't there be a bunch of requests for
 >> (potentially conflicting) additions?  When will it stop?

[Felix Wiemann]
 > The fact that there is one such a transformation doesn't mean we
 > will be adding anything, because there are IMO compelling reasons
 > for dash-transformation
...
 > which don't exist for the other transformations proposed in
 > alternatives.txt.

Compelling arguments could be put forth for any number of other
transformations.  docs/dev/rst/alternatives.txt doesn't list all
possible transformations, just a sampling.  I still think that once we
start down this path, it will be difficult to limit the uses of
character processing.  It will become a full-blown subsystem.  We must
be cautious.

 > (which are: monospace rendering, enterability and intuitiveness,
 > usualness in plain texts, and rather high unambiguousness),

Yes, those are compelling.  I'll change my vote to +0 (but read on).

 > Concerning the ellipsis, I'm not entirely sure if it's a good idea
 > to implement it, because it is not *that* important, sometimes the
 > transformation may be undesirable for some languages or
 > style-conventions,

Seems to me that *any* text transformation may be undesirable to
somebody, somewhere, sometime.

 > and it's not possible to escape special-cases because the logic
 > would be implemented in the LaTeX writer.

Why wouldn't the logic for ellipsis be in the parser?

 > Is escapability sufficient optionalness for you?

No.  That adds an extra burden for those people who *don't* want the
feature.  Better to make it a normally-disabled "power user" option
(or multiple options).  Then there's an expectation that the user will
know what they're getting into.

I say multiple options because there is no standard way to represent
the various dashes.  Some people use two hyphens for an em-dash (--),
some three (---).  According to `The Chicago Manual of Style`, two
hyphens is how typewritten manuscripts should represent an em-dash.
But we'd like to be able to represent an en-dash as well; 2-for-en and
3-for-em is convenient, but not universal.  Some people put spaces
around em-dashes --- like this --- and some don't---like this.
Typographically, the spaces are not correct and should be removed (at
least for common English usage---the mind boggles!).  Some people want
to distinguish em-dashes, but don't care about distinguishing between
en-dash & hyphen.

If we try to impose one set of conventions on all users, it will
inevitably conflict with someone's alternate conventions (not to
mention those who don't want any character processing at all!).  Even
if that is dismissed (reST is a markup language, after all), there are
variations in output requirements.

So these things would have to be options, and no, escaping doesn't cut
it.  Even options don't really cut it, because the processing is local
to the document, not the system on which it's being processed.  Pragma
directives would be ideal.

-- 
David Goodger <http://python.net/~goodger>

From: Felix Wiemann
> So the transformation would probably look like this:
>
> * Transform "---" to em-dash if it isn't preceded or followed by a =
dash.
> * Transform "--" to en-dash if it's surrounded by whitespace or by
>   alphanumeric characters.
> * No transformation takes place if one of the dashes is escaped.

Yuk. So <--> and <---> and <----> are all rendered differently? That's =
going to cause someone some grief.

Paul.


_________________________________________________________________________=
_
This e-mail and the documents attached are confidential and intended=20
solely for the addressee; it may also be privileged. If you receive this =

e-mail in error, please notify the sender immediately and destroy it.
As its integrity cannot be secured on the Internet, the Atos Origin =
group=20
liability cannot be triggered for the message content. Although the=20
sender endeavours to maintain a computer virus-free network, the sender=20
does not warrant that this transmission is virus-free and will not be=20
liable for any damages resulting from any virus transmitted.
_________________________________________________________________________=
_

I have written a patch that defines two new directives, namespace and
sysinclude.

namespace is useful when generating docbook output from multiple rst
files where some files would end up using the same internal ids.  It
works by prepending all ids with a user-defined string.  Typical usage
would be using a line like this ".. namespace:: foo" at the top of the
file.

sysinclude is a variant of the normal include directive, except that
it runs a command rather than reading a static file.  A trivial
example would be::

   The shells available on this system are:
   .. sysinclude:: /bin/cat /etc/shells
     :literal:

To avoid repetition of multiple similar commands, one can use a
:vardef: option to sysinclude to define a variable as "keyword value"
(in this case, sysinclude should not take any arguments).  This
variable is used in %(keyword)s expansions on the command that is
ran. It is also conventient to be able to refer to
environment-variables when specifying paths, thus we allow a
$ENV[NAME] syntax (it would be great if the normal include directive
also supported this).  For example::

  .. sysinclude::
    :vardef: ext_sqldoc $ENV[SCRIPTDIR]/ext_sqldoc.py --file $ENV[CEREBRUM_SRC]/design/core_tables.sql 
  
  Description of table entity_info
  ----------------------------------
  .. sysinclude:: %(ext_sqldoc)s --table entity_info

A patch against latest CVS is available at:
http://folk.uio.no/runefro/tmp/rst_ext.diff

It would be nice if this, or something similar, could be incorporated
in the official distribution.  The sysinclude directive is easy
(though cumbersome) to maintain as local variants of rst2, but the
namespace directive requires changes to docutils/nodes.py

Regards, 
-- 
Rune Frøysa

[Felix Wiemann]
 > With some more logic, it would be possible for grid tables too,
 > wouldn't it?

I don't think so.  Grid tables are ambiguous that way.  When
writing/drawing a grid table, I invariably include a left & right
margin in each table cell.  The spec specifically allows for such
margins, and the parser handles them transparently.  How is the text
in this cell aligned?

+------+
| word |
+------+

It's not centered!

[David Goodger]
 >> This wouldn't make sense for anything but single one-line
 >> paragraphs per cell.
 >
 > With "make sense", you mean "useful" or "syntactically possible"?

"Possible".  Counter-examples:

=====  =======================
row 1   if we try to make this
              right-aligned, it
           ends up looking like
               definition lists
             or bad indentation
-----  -----------------------
row 2     here's one paragraph

                 here's another

        the indentation above
        implies block quotes,
        not right alignment
=====  =======================

 >>> There has been a thread about adding classes for table rows and
 >>> columns, which would allow easier customization via CSS.  See
 >>> <http://thread.gmane.org/gmane.text.docutils.devel/1825>.
 >>
 >> Yes, that idea is workable too.
 >
 > By far not as straightforward and easy, because it requires CSS
 > tweaking.

It's a different use case, but a valid one.

-- 
David Goodger <http://python.net/~goodger>

Marcelo Huerta wrote:

> Felix Wiemann wrote:
>
>> (I.e., there won't be any problem with your dialog-example.)
>
> It would be problematic for the rendering. Inter-dialog observations
> are included in Spanish by inserting emdashes which *must* be in
> contact with the text in some part and separated in others; otherwise
> it's a syntactic error. For example:
>
> English version:
>
> "Yes,", he told me, "I must finish this work right now." I hated his
> stupid smile.
>
> Spanish version, -- instead of emdash:
>
> --Sí --me dijo él--, tengo que terminar este trabajo ya. --Odiaba su
> estúpida sonrisa.

In fact these dashes wouldn't be transformed, but they are all
en-dashes, not em-dashes.  I suppose you mean:

---Sí ---me dijo él---, tengo que terminar este trabajo ya. ---Odiaba su
estúpida sonrisa.

These dash-groups are all correctly transformed to em-dashes.

-- 
When replying to my email address, please ensure
that the mail header contains 'Felix Wiemann'.

http://www.ososo.de/

El 18/10/2004 a las 16:21, Felix Wiemann <Fel...@gm...> dijo=
,
en su mensaje "[Docutils-users] Re: Rendering emdashes":

> (I.e., there won't be any problem with your dialog-example.)

It would be problematic for the rendering. Inter-dialog observations
are included in Spanish by inserting emdashes which *must* be in
contact with the text in some part and separated in others; otherwise
it's a syntactic error. For example:

English version:

"Yes,", he told me, "I must finish this work right now." I hated his
stupid smile.

Spanish version, -- instead of emdash:

--S=ED --me dijo =E9l--, tengo que terminar este trabajo ya. --Odiaba=
 su
est=FApida sonrisa.

--=20
                    o-=3D< Marcelo >=3D-o

carrancho. Carancho que vive en chozas.
  --Del "Bichonario" (Gim=E9nez/Wright)

<chr...@e-...> wrote:

> Felix Wiemann wrote:
>
>> With sufficient intelligence, it will do in 99,9%.  (I can't remember
>> any case where the effect would have been undesired.)  Is escapability
>> sufficient optionalness for you?
>
> 99,9% ist not really enough. Any experience of a single person
> (including my own experience) does not mean there are not any cases
> one thinks they should not be

That's what escapes are for.

> or not even that there are only a few.

We have to rely on some judging.  We're also assuming that there are few
real-life cases where words end with one or two underscores (which may
very well happen in programming-related documents).

> IMHO docutils already has enough markup to remember and including any
> and everything in another markup makes it really difficult to write.

On the other hand one can argue that remembering to enter a real en-dash
(and how to do so) is difficult too.  I find that a much more practical
and realistic reason than possible clashes with a double-dash
transformation.

> So the only way to include even more special markup cases (which an
> automatic translation of "x---x" or "x -- x" would be) may be to reuse
> an existing markup facility of ReST. I am not familiar enough with the
> markup rules but I thought of something like the use of a trailing
> "__" which is used for some markup stuff. Is there some markup which
> is extensible with something like a directive like "...<ellipsis>__"?

This lacks:

* readability and unobtrusiveness
* intuitive enterability

> BTW, a program listing (Java, Javascript etc) does contain quite often
> something like avariable--;

This is no problem, because:

* Even in normal text, "avariable--;" would not be transformed.
* Program listings are usually contained in literal blocks, which won't
  be subject to any transformation anyway.

Which just reminds me of parsed literals: There, the dash-transformation
is done (with my example patch), but it's probably undesired, because
the text is rendered in monospaced font.  This problem has yet to be
solved.

-- 
When replying to my email address, please ensure
that the mail header contains 'Felix Wiemann'.

http://www.ososo.de/

Marcelo Huerta wrote:

> My intention was to say that, convenient as I might find the "---"
> shortcut (and I would really prefer "--", as it's our usual way to
> replace the em dash when writing a text file, en dashes being written
> simply as "-"),

No.  There is a difference between dash and en-dash (which is very
important for German, e.g.) which would be lost if en-dashes were
written as single dashes ("-").  "--" for en-dash and "---" for em-dash
is also the way LaTeX does it.

> I wonder how could be easily implemented to avoid inconvenience to a
> Spanish language writer.  [E.g. "---He reñido a un posadero."]

After some googling it looks like some people prefer using "foo --- bar"
instead of "foo---bar" in normal text, so there probably shouldn't be
any requirement about leading or trailing alphanumeric characters for
em-dashes anyway.  (I.e., there won't be any problem with your
dialog-example.)

Something I forgot in my earlier postings is that the en-dash may also
be needed for sequences ("pp. 15--18"), or compound expressions if one
of the components contains spaces ("post--World War 1").

So the transformation would probably look like this:

* Transform "---" to em-dash if it isn't preceded or followed by a dash.
* Transform "--" to en-dash if it's surrounded by whitespace or by
  alphanumeric characters.
* No transformation takes place if one of the dashes is escaped.

To give an example-implementation of what I mean:

------------------------------------------------------------------------------
--- docutils/parsers/rst/states.py.~1.78.~	2004-10-15 14:58:05.000000000 +0200
+++ docutils/parsers/rst/states.py	2004-10-18 21:00:12.000000000 +0200
@@ -483,7 +483,10 @@
         method, which enables additional interpreted text roles.
         """
 
-        self.implicit_dispatch = [(self.patterns.uri, self.standalone_uri),]
+        self.implicit_dispatch = [
+            (self.patterns.uri, self.standalone_uri),
+            (self.patterns.en_dash, lambda x, y: [nodes.Text(u'\u2013')]),
+            (self.patterns.em_dash, lambda x, y: [nodes.Text(u'\u2014')])]
         """List of (pattern, bound method) tuples, used by
         `self.implicit_inline`."""
 
@@ -680,7 +683,27 @@
                 r"""
                 %(start_string_prefix)s
                 (RFC(-|\s+)?(?P<rfcnum>\d+))
-                %(end_string_suffix)s""" % locals(), re.VERBOSE))
+                %(end_string_suffix)s""" % locals(), re.VERBOSE),
+          en_dash=re.compile(
+              r"""
+              (
+                (?<!\S)   # leading whitespace or nothing
+                --
+                (?=\s|\000[ \n]|$)   # trailing whitespace (possibly
+                                     # escaped) or nothing
+              |
+                (?<=\w)   # leading alphanumeric character
+                --
+                (?=\000?\w)   # trailing alphanumeric character,
+                              # possibly escaped
+              )
+              """, re.VERBOSE | re.UNICODE),
+          em_dash = re.compile(
+              r"""
+              (?<![\000-])   # No leading escape or dash.
+              ---
+              (?!-)   # No trailing dash.
+              """, re.VERBOSE))
 
     def quoted_start(self, match):
         """Return 1 if inline markup start-string is 'quoted', 0 if not."""
------------------------------------------------------------------------------

Some examples of what is transformed and what is not transformed (later,
we could also use this for testing if the patch is accepted):

Transformed Dashes
==================

En-dashes
---------

Foo -- bar, 10--20, foo--bar, foo\ --\ bar, foo--\bar.

-- at the beginning, at the end --

--

Em-dashes
---------

Foo --- bar, foo---bar, foo ---bar, foo--- bar, foo---\bar, -\ ---\ -,
foo/---bar, foo---/bar, foo---\\bar, foo\\---\\bar.

---at the beginning, at the end---

---

Untransformed Dashes
====================

En-dashes
---------

Foo-- bar, foo --bar, foo\--bar, foo-\-bar, foo--\ bar, foo/--bar,
foo--/bar, foo/--/bar, "--foo", "bar--", \\--foo, bar--\\.

--at the beginning, at the end--

Em-dashes
---------

Foo----bar, foo-----bar, foo\---bar, foo-\--bar, foo--\-bar.

-- 
When replying to my email address, please ensure
that the mail header contains 'Felix Wiemann'.

http://www.ososo.de/

El 18/10/2004 a las 02:08, David Goodger <go...@py...> dijo, e=
n
su mensaje "[Docutils-users] Rendering emdashes (Was: Re: rendering
ellipsis)":

> Marcelo, could you please clarify: in Spanish, is dialogue written
> with three dashes, or with one em dash?

I meant an em dash, of course. Sorry for not being clear. My intentio=
n
was to say that, convenient as I might find the "---" shortcut (and I
would really prefer "--", as it's our usual way to replace the em das=
h
when writing a text file, en dashes being written simply as "-"), I
wonder how could be easily implemented to avoid inconvenience to a
Spanish language writer.

--=20
                    o-=3D< Marcelo >=3D-o

cacharro. Animal joven para contener l=EDquidos.
  --Del "Bichonario" (Gim=E9nez/Wright)

hi,

> With sufficient intelligence, it will do in 99,9%.  (I can't remember
> any case where the effect would have been undesired.)  Is escapability
> sufficient optionalness for you?

just my opinion as an outsider and seldom user of docutils:

99,9% ist not really enough. Any experience of a single person =
(including my own experience) does not mean there are not any cases one =
thinks they should not be or not even that there are only a few. In =
almost all projects I've done there was a point that something I assumed =
always true was false in some cases and a more generalized approach was =
necessary...

IMHO docutils already has enough markup to remember and including any =
and everything in another markup makes it really difficult to write.=20
Being able to escape this markup might be an option but being forced to =
escape these cases also means to know and be aware of that such an =
implicit markup does exist. The other way round is much better I think.

So the only way to include even more special markup cases (which an =
automatic translation of "x---x" or "x -- x" would be) may be to reuse =
an existing markup facility of ReST. I am not familiar enough with the =
markup rules but I thought of something like the use of a trailing "__" =
which is used for some markup stuff. Is there some markup which is =
extensible with something like a directive like "...<ellipsis>__"?


BTW, a program listing (Java, Javascript etc) does contain quite often =
something like=20
	avariable--;
which is exactly what you said would not happen often...

only what I think...
thanks=20
chris

[Marcelo Huerta]
> How do you would address the converse situation, meaning, you *need*
> to convert "triple dash + nonspace" into "emdash + nonspace"? That's the
> way dialogs are written in Spanish. It would be *extremely*
> inconvenient to have to escape an space for each line of dialog, for
> example.
> 
>  ---He reñido a un posadero.
>  ---¿Por qué? ¿Cuándo? ¿Dónde? ¿Cómo?
>  ---Porque cuando donde como sirven mal, me desespero.

Marcelo, could you please clarify: in Spanish, is dialogue written 
with three dashes, or with one em dash?

Thanks.

-- David Goodger