docutils-users Mailing List for Docutils: Documentation Utilities

On Fri, May 31, 2013 at 3:55 AM, Christophe de Vienne
<cde...@gm...> wrote:
> Hi,
>
> I am trying to set a custom class on an anchor.
> The best I could do for now is :
>
> .. class:: someclass
>
> `mylink`
>
>
> This generates something like :
>
> <p class="someclass"><a class="external" href="">mylink</a></p>
>
>
> But what I really want is:
>
> <p><a class="someclass external" href="">mylink</a></p>
>
>
> Is there a way to do that ?

Short answer: no.

Short workaround: do what you show above ("class" directive), and in
your stylesheet, apply a style to an <a> element *within* an element
with class="someclass". It's the easiest way to do what you want.

Long workaround, but no better:

$ ./tools/rst2pseudoxml.py << 'EOF'
> .. role:: myclass
> .. |link| replace:: :myclass:`hyperlink`
> .. _link: http://example.org
>
> Some text with a |link|_.
>
> EOF

Resulting relevant pseudo-XML:

    <paragraph>
        Some text with a
        <reference refuri="http://example.org">
            <inline classes="myclass">
                hyperlink
        .

Equivalent HTML:

<p>Some text with a <a class="reference external"
href="http://example.org"><span
class="myclass">hyperlink</span></a>.</p>

Note that the class is applied to the text *inside* the reference, not
to the <a> tag itself. Which makes for another interesting style
requirement.

--
David Goodger <http://python.net/~goodger>

On 2013-05-31, Christophe de Vienne wrote:
> Hi,

> I am trying to set a custom class on an anchor.
> The best I could do for now is :

> .. class:: someclass

> `mylink`

> This generates something like :

><p class="someclass"><a class="external" href="">mylink</a></p>

Yes, the class argument is put on the following block-level element, in this
case a paragraph.

> But what I really want is:

><p><a class="someclass external" href="">mylink</a></p>

> Is there a way to do that ?

I don't know of such a way short of defining a custom role in Python.

The general way to get a class on an inline element (or text portion) is a
custom role. Unfortunately, you cannot nest inline markup, so that ::

  .. role:: muffle

  :muffle:`was_`

results in the document structure ::

  <document source="/tmp/foo.rst">
      <paragraph>
          <inline classes="muffle">
              was_

It should be possible, to code in Python a new, named reStructureText
role (similar to the standard roles like "subscript" or "math") that uses
the content as hyperlink target. 

Lets call this hypothetical role "hyperlink".
It would work like standard hyperlink syntax, i.e. `wolle molle`_ is equall to
:hyperlink:`wolle molle`.

The advantage is, that you can use it in a role definition directive as base
role::

  .. role:: muffle(hyperlink)
     :class: someclass
     
  :muffle:`http://example.org/`

would then result in

    <paragraph>
        <reference class="someclass", refuri="http://example.org">
            http://example.org

(untested).

Günter

Hi,

I am trying to set a custom class on an anchor.
The best I could do for now is :

.. class:: someclass

`mylink`


This generates something like :

<p class="someclass"><a class="external" href="">mylink</a></p>


But what I really want is:

<p><a class="someclass external" href="">mylink</a></p>


Is there a way to do that ?

Thanks,

Christophe

PS: I am not subscribed to the list, please CC me.

On 2013-05-28, Valentin Haenel wrote:
> Hi,

> I am currently working on a custom writer:

> https://github.com/ipython/nbconvert/blob/master/rst2ipynb.py

> And inside the writer I need to get at the path of the original input file. Is
> there any way to do this is docutils?

> Thanks in advance for your advice.

Several nodes have a "source" argument. 
docutils.utils has a function to get source and source line::

 def get_source_line(node):
    """
    Return the "source" and "line" attributes from the `node` given or from
    its closest ancestor.
    """
    while node:
        if node.source or node.line:
            return node.source, node.line
        node = node.parent
    return None, None


Günter

On 2013-05-28, Valentin Haenel wrote:
> Hi again,

> I encountered some more issues with:

> https://github.com/ipython/nbconvert/blob/master/rst2ipynb.py

> on the following sphinx document:

This is the wrong list. While Sphinx uses Docutils and reStructuredText,
extensions to this extension are better discussed on sphinx-users.

Günter

Hi again,

I encountered some more issues with:

https://github.com/ipython/nbconvert/blob/master/rst2ipynb.py

on the following sphinx document:

https://github.com/scipy-lectures/scipy-lecture-notes/blob/master/intro/numpy/array_object.rst

I get the following traceback:

zsh» ~/git-working/nbconvert/rst2ipynb.py --traceback array_object.rst
array_object.rst:5: (ERROR/3) Unknown directive type "currentmodule".

.. currentmodule:: numpy

array_object.rst:215: (ERROR/3) Unknown interpreted text role "mod".
array_object.rst:369: (ERROR/3) Unknown directive type "plot".

.. plot:: pyplots/numpy_intro_1.py

array_object.rst:380: (ERROR/3) Unknown directive type "plot".

.. plot:: pyplots/numpy_intro_2.py

array_object.rst:382: (ERROR/3) Unknown interpreted text role "ref".
array_object.rst:412: (ERROR/3) Unknown interpreted text role "ref".
array_object.rst:665: (ERROR/3) Unknown interpreted text role "ref".
Traceback (most recent call last):
  File "/home/esc/git-working/nbconvert/rst2ipynb.py", line 21, in <module>
    description=description)
  File "/home/esc/anaconda/lib/python2.7/site-packages/docutils/core.py", line 352, in publish_cmdline
    config_section=config_section, enable_exit_status=enable_exit_status)
  File "/home/esc/anaconda/lib/python2.7/site-packages/docutils/core.py", line 218, in publish
    self.apply_transforms()
  File "/home/esc/anaconda/lib/python2.7/site-packages/docutils/core.py", line 199, in apply_transforms
    self.document.transformer.apply_transforms()
  File "/home/esc/anaconda/lib/python2.7/site-packages/docutils/transforms/__init__.py", line 171, in apply_transforms
    transform.apply(**kwargs)
  File "/home/esc/anaconda/lib/python2.7/site-packages/docutils/transforms/writer_aux.py", line 86, in apply
    title = nodes.title('', language.labels[node_name])
KeyError: 'seealso'

It can be fixed with the following patch:

zsh» gid
diff --git i/docutils/docutils/transforms/writer_aux.py w/docutils/docutils/transforms/writer_aux.py
index 9e9a65f935..c58b55ec4c 100644
--- i/docutils/docutils/transforms/writer_aux.py
+++ w/docutils/docutils/transforms/writer_aux.py
@@ -83,6 +83,9 @@ class Admonitions(Transform):
                 # Specific admonition.  Transform into a generic admonition.
                 admonition = nodes.admonition(node.rawsource, *node.children,
                                               **node.attributes)
-                title = nodes.title('', language.labels[node_name])
+                try:
+                    title = nodes.title('', language.labels[node_name])
+                except KeyError:
+                    title = nodes.title('', node_name)
                 admonition.insert(0, title)
                 node.replace_self(admonition)

If you have to reproduce, please use the git versions from:

https://github.com/esc/nbconvert/tree/improve_rst2ipynb

and

https://github.com/bebraw/pypandoc

Maybe someone who knows the code a little better can send me in the
right direction, on the 'proper' way to fix this.

V-

Hi,

I am currently working on a custom writer:

https://github.com/ipython/nbconvert/blob/master/rst2ipynb.py

And inside the writer I need to get at the path of the original input
file. Is there any way to do this is docutils?

Thanks in advance for your advice.

Valentin

On 2013-05-24, 13718354131 wrote:

Dear 13718354131,

> i found that in  restructured text search function:

> 1.can not search chinese character

> 2.can not search  short words


It seems you posted to the wrong list.

There is no search function in reStructuredText (the document format).
http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html

There is no search function in Docutils (the reStructuredText to
HTML/LaTeX,ODT,... converter).
http://docutils.sourceforge.net/docs/

May be your problem is with the Search implemented by Sphinx (another
reStructuredText to HTML/PDF converter built on Docutils).
http://sphinx-doc.org
Please ask at the list there.


Günter

On 05/22/2013 09:57 PM, David Goodger wrote:
> On Wed, May 22, 2013 at 8:17 AM, Peter L. Soendergaard
> <pe...@so...> wrote:
>> Hi,
>>
>> I am running docutils.core.publish_string inside some python scripts and
>> I need to process a lot of REST files from other people, and they often
>> contains errors. I need to do some preprocessing to the files before
>> passing them onto docutils, so that is why I call publish_string from
>> inside of Python instead of the command line tools.
>>
>> Currently, I just see errors and warnings on the command line like:
>>
>> <string>:73: (ERROR/3) Unexpected indentation.
>> <string>:74: (WARNING/2) Block quote ends without a blank line;
>> unexpected unindent.
>>
>> Are there some parameters that I can pass onto publish_string so that I
>> can capture the error output instead of it going straight to stderr?
>>
>> Or some better method than calling publish_string ?
> publish_string seems like the right API for your use case.
>
> You can capture the text of the system messages by assigning a
> file-like object to the "warning_stream" setting. E.g. a
> StringIO.StringIO object or a custom object with a "write" method.
> This stream is used by docutils.utils.Reporter; the stream's write
> method is called once per error/warning. <stderr> is the default if no
> alternate stream is passed in.
>
> Or you could set "halt_level" appropriately (and "traceback" to True)
> to catch exceptions in try/except blocks.
>
> See http://docutils.sourceforge.net/docs/user/config.html
>
> Tip: the "source_path" parameter to publish_string will let you pass
> the source text's filename/path, which will then be reported in the
> system messages (currently only "<string>", because Docutils doesn't
> know where the text came from).
>
> --
> David Goodger <http://python.net/~goodger>
Thanks, this was just what I needed. I will start working on it.

Cheers,
Peter.

On Wed, May 22, 2013 at 3:40 PM, Michael Prisant
<mic...@gm...> wrote:
> Appreciate the explanation of source path configuration which I hadn't fully
> understand. And the summary of various modalities for error handling in a
> program setting as well as directly answering the post.
>
> The indentation which currently is pretty easy to identify/correct in
> individual reST text documents is harder to deal when batch processing many
> documents from multiple authors.  One way or another the errors are
> identified when the text source is handed off to the docutils library.  But
> has someone authored a batch corrector/reformator for multiple reST text
> source with some common author errors like indentation?

I don't know of any such tool. A perfect tool is, I believe,
impossible. The indentation depends on the intent of the author, and
multiple indentations are possible at almost any point in the text
(e.g. block quotes, definition lists). Perhaps a "good enough" tool
may be possible.

-- DG

> On Wed, May 22, 2013 at 3:57 PM, David Goodger <go...@py...> wrote:
>>
>> On Wed, May 22, 2013 at 8:17 AM, Peter L. Soendergaard
>> <pe...@so...> wrote:
>> > Hi,
>> >
>> > I am running docutils.core.publish_string inside some python scripts and
>> > I need to process a lot of REST files from other people, and they often
>> > contains errors. I need to do some preprocessing to the files before
>> > passing them onto docutils, so that is why I call publish_string from
>> > inside of Python instead of the command line tools.
>> >
>> > Currently, I just see errors and warnings on the command line like:
>> >
>> > <string>:73: (ERROR/3) Unexpected indentation.
>> > <string>:74: (WARNING/2) Block quote ends without a blank line;
>> > unexpected unindent.
>> >
>> > Are there some parameters that I can pass onto publish_string so that I
>> > can capture the error output instead of it going straight to stderr?
>> >
>> > Or some better method than calling publish_string ?
>>
>> publish_string seems like the right API for your use case.
>>
>> You can capture the text of the system messages by assigning a
>> file-like object to the "warning_stream" setting. E.g. a
>> StringIO.StringIO object or a custom object with a "write" method.
>> This stream is used by docutils.utils.Reporter; the stream's write
>> method is called once per error/warning. <stderr> is the default if no
>> alternate stream is passed in.
>>
>> Or you could set "halt_level" appropriately (and "traceback" to True)
>> to catch exceptions in try/except blocks.
>>
>> See http://docutils.sourceforge.net/docs/user/config.html
>>
>> Tip: the "source_path" parameter to publish_string will let you pass
>> the source text's filename/path, which will then be reported in the
>> system messages (currently only "<string>", because Docutils doesn't
>> know where the text came from).
>>
>> --
>> David Goodger <http://python.net/~goodger>
>>
>>
>> ------------------------------------------------------------------------------
>> Try New Relic Now & We'll Send You this Cool Shirt
>> New Relic is the only SaaS-based application performance monitoring
>> service
>> that delivers powerful full stack analytics. Optimize and monitor your
>> browser, app, & servers with just a few lines of code. Try New Relic
>> and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may
>> _______________________________________________
>> Docutils-users mailing list
>> Doc...@li...
>> https://lists.sourceforge.net/lists/listinfo/docutils-users
>>
>> Please use "Reply All" to reply to the list.
>
>
>
>
> --
> Michael G. Prisant

Appreciate the explanation of source path configuration which I hadn't
fully understand. And the summary of various modalities for error handling
in a program setting as well as directly answering the post.

The indentation which currently is pretty easy to identify/correct in
individual reST text documents is harder to deal when batch processing many
documents from multiple authors.  One way or another the errors are
identified when the text source is handed off to the docutils library.  But
has someone authored a batch corrector/reformator for multiple reST text
source with some common author errors like indentation?

Michael

On Wed, May 22, 2013 at 3:57 PM, David Goodger <go...@py...> wrote:

> On Wed, May 22, 2013 at 8:17 AM, Peter L. Soendergaard
> <pe...@so...> wrote:
> > Hi,
> >
> > I am running docutils.core.publish_string inside some python scripts and
> > I need to process a lot of REST files from other people, and they often
> > contains errors. I need to do some preprocessing to the files before
> > passing them onto docutils, so that is why I call publish_string from
> > inside of Python instead of the command line tools.
> >
> > Currently, I just see errors and warnings on the command line like:
> >
> > <string>:73: (ERROR/3) Unexpected indentation.
> > <string>:74: (WARNING/2) Block quote ends without a blank line;
> > unexpected unindent.
> >
> > Are there some parameters that I can pass onto publish_string so that I
> > can capture the error output instead of it going straight to stderr?
> >
> > Or some better method than calling publish_string ?
>
> publish_string seems like the right API for your use case.
>
> You can capture the text of the system messages by assigning a
> file-like object to the "warning_stream" setting. E.g. a
> StringIO.StringIO object or a custom object with a "write" method.
> This stream is used by docutils.utils.Reporter; the stream's write
> method is called once per error/warning. <stderr> is the default if no
> alternate stream is passed in.
>
> Or you could set "halt_level" appropriately (and "traceback" to True)
> to catch exceptions in try/except blocks.
>
> See http://docutils.sourceforge.net/docs/user/config.html
>
> Tip: the "source_path" parameter to publish_string will let you pass
> the source text's filename/path, which will then be reported in the
> system messages (currently only "<string>", because Docutils doesn't
> know where the text came from).
>
> --
> David Goodger <http://python.net/~goodger>
>
>
> ------------------------------------------------------------------------------
> Try New Relic Now & We'll Send You this Cool Shirt
> New Relic is the only SaaS-based application performance monitoring service
> that delivers powerful full stack analytics. Optimize and monitor your
> browser, app, & servers with just a few lines of code. Try New Relic
> and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may
> _______________________________________________
> Docutils-users mailing list
> Doc...@li...
> https://lists.sourceforge.net/lists/listinfo/docutils-users
>
> Please use "Reply All" to reply to the list.
>



-- 
Michael G. Prisant
 <Mic...@gm...>

On Wed, May 22, 2013 at 8:17 AM, Peter L. Soendergaard
<pe...@so...> wrote:
> Hi,
>
> I am running docutils.core.publish_string inside some python scripts and
> I need to process a lot of REST files from other people, and they often
> contains errors. I need to do some preprocessing to the files before
> passing them onto docutils, so that is why I call publish_string from
> inside of Python instead of the command line tools.
>
> Currently, I just see errors and warnings on the command line like:
>
> <string>:73: (ERROR/3) Unexpected indentation.
> <string>:74: (WARNING/2) Block quote ends without a blank line;
> unexpected unindent.
>
> Are there some parameters that I can pass onto publish_string so that I
> can capture the error output instead of it going straight to stderr?
>
> Or some better method than calling publish_string ?

publish_string seems like the right API for your use case.

You can capture the text of the system messages by assigning a
file-like object to the "warning_stream" setting. E.g. a
StringIO.StringIO object or a custom object with a "write" method.
This stream is used by docutils.utils.Reporter; the stream's write
method is called once per error/warning. <stderr> is the default if no
alternate stream is passed in.

Or you could set "halt_level" appropriately (and "traceback" to True)
to catch exceptions in try/except blocks.

See http://docutils.sourceforge.net/docs/user/config.html

Tip: the "source_path" parameter to publish_string will let you pass
the source text's filename/path, which will then be reported in the
system messages (currently only "<string>", because Docutils doesn't
know where the text came from).

--
David Goodger <http://python.net/~goodger>

Sorry this link should have been in the previous email:

http://stackoverflow.com/questions/2077897/substitute-multiple-whitespace-with-single-whitespace-in-python

And especially note this repsonse:

A simple possibility (if you'd rather avoid REs) is

' '.join(mystring.split())

The split and join perform the task you're explicitly asking about -- plus,
they also do the extra one that you don't talk about but is seen in your
example, removing trailing spaces;-).
  share <http://stackoverflow.com/a/2077944>|improve this
answer<http://stackoverflow.com/posts/2077944/edit>
  answered Jan 16 '10 at 15:54
 <http://stackoverflow.com/users/95810/alex-martelli>
 Alex Martelli <http://stackoverflow.com/users/95810/alex-martelli>
261k27458810

    1
Oh cool, I was fumbling with a similar solution, but using split(' ') and
then a filter to remove empty elements. I never knew split with no
arguments worked like this. This is also much faster, timeit.py gives me
around 0.74usec for this, versus 5.75usec for regular expressions. – Roman
Stolper <http://stackoverflow.com/users/217337/roman-stolper> Jan 16 '10 at
16:00<http://stackoverflow.com/questions/2077897/substitute-multiple-whitespace-with-single-whitespace-in-python#comment2008468_2077944>
  1
@Roman, yes, x.split() (and x.split(None)) splits on *sequences of
whitespace* (including tabs, newlines, etc, like re's \s) of length 1+ --
and it's pretty fast indeed. So, always glad to help! – Alex
Martelli<http://stackoverflow.com/users/95810/alex-martelli> Jan
16 '10 at 16:25<http://stackoverflow.com/questions/2077897/substitute-multiple-whitespace-with-single-whitespace-in-python#comment2008566_2077944>



On Wed, May 22, 2013 at 9:41 AM, Michael Prisant
<mic...@gm...>wrote:

> Perhaps just identify and rectify the indentation/tabbing errors in the
> preprocessing by using python string split to first split the reST string
> source into lines  and then using python string replace to correct
> mistabbing.
>
> Don't know of a publish method option for this but would be great if one
> existed.  Sort of need a reST source string "lint"
>
> MIchael
>
> PS Ditto to being occasionally vexed by the seeming need for precise and
> consistent indentation in reST source for error free publishing .  Emacs
> handles this for me but like you will have to implement
> checking/rectification for documents prepared for others
>
>
>
> On Wed, May 22, 2013 at 9:17 AM, Peter L. Soendergaard <
> pe...@so...> wrote:
>
>> Hi,
>>
>> I am running docutils.core.publish_string inside some python scripts and
>> I need to process a lot of REST files from other people, and they often
>> contains errors. I need to do some preprocessing to the files before
>> passing them onto docutils, so that is why I call publish_string from
>> inside of Python instead of the command line tools.
>>
>> Currently, I just see errors and warnings on the command line like:
>>
>> <string>:73: (ERROR/3) Unexpected indentation.
>> <string>:74: (WARNING/2) Block quote ends without a blank line;
>> unexpected unindent.
>>
>> Are there some parameters that I can pass onto publish_string so that I
>> can capture the error output instead of it going straight to stderr?
>>
>> Or some better method than calling publish_string ?
>>
>> Cheers,
>> Peter.
>>
>>
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Try New Relic Now & We'll Send You this Cool Shirt
>> New Relic is the only SaaS-based application performance monitoring
>> service
>> that delivers powerful full stack analytics. Optimize and monitor your
>> browser, app, & servers with just a few lines of code. Try New Relic
>> and get this awesome Nerd Life shirt!
>> http://p.sf.net/sfu/newrelic_d2d_may
>> _______________________________________________
>> Docutils-users mailing list
>> Doc...@li...
>> https://lists.sourceforge.net/lists/listinfo/docutils-users
>>
>> Please use "Reply All" to reply to the list.
>>
>
>
>
> --
> Michael G. Prisant
>  <Mic...@gm...>




-- 
Michael G. Prisant
 <Mic...@gm...>

Perhaps just identify and rectify the indentation/tabbing errors in the
preprocessing by using python string split to first split the reST string
source into lines  and then using python string replace to correct
mistabbing.

Don't know of a publish method option for this but would be great if one
existed.  Sort of need a reST source string "lint"

MIchael

PS Ditto to being occasionally vexed by the seeming need for precise and
consistent indentation in reST source for error free publishing .  Emacs
handles this for me but like you will have to implement
checking/rectification for documents prepared for others


On Wed, May 22, 2013 at 9:17 AM, Peter L. Soendergaard
<pe...@so...>wrote:

> Hi,
>
> I am running docutils.core.publish_string inside some python scripts and
> I need to process a lot of REST files from other people, and they often
> contains errors. I need to do some preprocessing to the files before
> passing them onto docutils, so that is why I call publish_string from
> inside of Python instead of the command line tools.
>
> Currently, I just see errors and warnings on the command line like:
>
> <string>:73: (ERROR/3) Unexpected indentation.
> <string>:74: (WARNING/2) Block quote ends without a blank line;
> unexpected unindent.
>
> Are there some parameters that I can pass onto publish_string so that I
> can capture the error output instead of it going straight to stderr?
>
> Or some better method than calling publish_string ?
>
> Cheers,
> Peter.
>
>
>
>
>
>
> ------------------------------------------------------------------------------
> Try New Relic Now & We'll Send You this Cool Shirt
> New Relic is the only SaaS-based application performance monitoring service
> that delivers powerful full stack analytics. Optimize and monitor your
> browser, app, & servers with just a few lines of code. Try New Relic
> and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may
> _______________________________________________
> Docutils-users mailing list
> Doc...@li...
> https://lists.sourceforge.net/lists/listinfo/docutils-users
>
> Please use "Reply All" to reply to the list.
>



-- 
Michael G. Prisant
 <Mic...@gm...>

Hi,

I am running docutils.core.publish_string inside some python scripts and 
I need to process a lot of REST files from other people, and they often 
contains errors. I need to do some preprocessing to the files before 
passing them onto docutils, so that is why I call publish_string from 
inside of Python instead of the command line tools.

Currently, I just see errors and warnings on the command line like:

<string>:73: (ERROR/3) Unexpected indentation.
<string>:74: (WARNING/2) Block quote ends without a blank line; 
unexpected unindent.

Are there some parameters that I can pass onto publish_string so that I 
can capture the error output instead of it going straight to stderr?

Or some better method than calling publish_string ?

Cheers,
Peter.

On 2013-05-14, Michael Prisant wrote:

> But my specific question remains.

It actually makes sense to split your question into several ones:

a) How to write a custom directive?
b) How to get specific output for html and latex from the same source?

> How do I write python code in the proper "Docutilic idiom" perhaps
> resembling my start with

> class BibDirective(Directive):
>     has_content = True
>     def run(self):
> ...
> directives.register_directive("bibentries", BibDirective)

> which will allow me to translate the content to both html and latex.

How to write a custom directive
===============================

* Get a clear understanding of the intended input and output.

* Try to "reverse-engineer": write down "explicit" rst input that results in
  the desired output.

  For example, let us assume the rst-source code ::

    .. bibentries:: database

       BCC+87;;

  should produce the same result as::

    .. [fntguide] LaTeX3 Project Team, `LaTeX2ε font selection`, 2005.
       http://mirror.ctan.org/macros/latex/doc/fntguide.pdf
    .. [encguide] Frank Mittelbach, Robin Fairbairns, Werner Lemberg,
       LaTeX3 Project Team, `LaTeX font encodings`, 2006.
       http://mirror.ctan.org/macros/latex/doc/encguide.pdf
    .. [greek-usage] Apostolos Syropoulos, `Writing Greek with the greek option
       of the babel package`, 1997.

* Translating the "rst representation" of the desired output with rst2xml or
  rst2pseudoxml, you can find out how to represent the desird output in
  terms of Docutils' document model (the "doctree").

  Hints:

  - Try to find the most appropriate doctree element for the purpose.

  - Use class arguments to enhance the "stock" doctree elements with
    additional semantics.

  - It may come out that you do not need a custom directive at all,
    especially if the rst source is generated. Remember that custom
    directives make the rst source non-portable.

  - Only in cases where there is no Docutils equivalent of the desired
    output or you already have a string with the desired output in the
    target format(s) at hand, you may ressort to "raw" input.

    Warning: you need one raw node for every to-be-supported output
    format::

      .. raw:: html

         <p>This is raw HTML</p>

      .. raw:: latex

         This is raw  \LaTeX.

* Model your directive on the source code that generates the required nodes.

  - a directive may return more than one node

  - you may have to browse large parts of docutils/parsers/rst/,
    `grep` is your friend.


---------------------------------------------------------------------------

This is the first outline. Feel free to fill it in with more examples and
concrete tips.

Question b)

> So for purposes of explanation lets try for '<pre> This is the html
> bibdirective </pre>' to be emitted for html
...
> and '\begin{verbatim} This is the latex bibdirective \end{verbatim}' to be
> emitted
for latex:

Assuming only translation to HTML and LaTeX, the rst source for this
could be::

  .. class:: no-latex
  
  ::
  
    This is the html bibdirective.
    
  .. class:: no-html
  
  ::
  
    This is the latex bibdirective.

with the docutils.conf file containing::

  [html4css1 writer]
  strip-elements-with-classes: no-html
  
  [latex2e writer]
  # skip objects with the "no-latex" class
  strip-elements-with-classes: no-latex

This method can, of course, also be applied to parts of the output::

  .. role:: no-html
  .. role:: no-latex
  
  .. parsed-literal::
  
    This is the :no-html:`latex`\ :no-latex:`html` bibdirective.

The "raw" directive and role work the other way round: it is ignored by all
writers exept the specified one (similar to the `only...` directives in
Sphinx).

> Yes this is a dumb example but I think it is enough to get started. How can
> this question be made more simple or specific? 

I hope this can get you started.

For real code examples, it is best if you provide a `minimal working
example`__. In this case, this may be a Python file that imports docutils
and contains your custom directive, a sample input string, a sample output
string, code to call the Publisher, and a comparison of expected and actual
output. Leave out all parts that are irrelevant for the stated problem.
With such a file, the developers could play and test the result
without the need to (re)create the complete framework.

__ http://docutils.sourceforge.net/BUGS.html#minimal-example


For your task, you may also have a look at other bibliography add-ons,
e.g. http://code.google.com/p/bibstuff/

See also "Footnote & Citation Gathering" in the TODO list:
http://docutils.sourceforge.net/docs/dev/todo.html#unimplemented-transforms


Günter

Thanks, sorry dumb error on my part. This works!

Thanks for coding this feature.  The latex writer and the rst2latex command
line properly translates citations in the reST source for BibTeX processing
in the latex output.

Cool and very useful! (At least for me)

Michael


On Wed, May 15, 2013 at 10:34 AM, Guenter Milde <mi...@us...> wrote:

> On 2013-05-13, Michael Prisant wrote:
>
> ...
>
> > But it looks like the writer is evaluating self.bibtex as null.
>
> null? bool(self.bibtex) == False, because self.bibtex == '' # empty string
>
> > I thought that I was setting this correctly by setting: (all the other
> > settings seem to push through correctly to the latex writer with
> > appropriate changes when set to different values.
>
> > ltx_overrides = {
> >     'input_encoding': 'utf8',
> >     'output_encoding': 'utf8',
> >     'doctitle_xform': True,
> >     'initial_header_level': 2,
> >     'use-latex_abstract': True,
> >     'use_latex_citations': True,
> >     'use_bibtex': 'no_abstract,Articles,Books,Theses',
> >     'template': 'my_default.tex',
> >     'use_bibtex': '',
>       ^^^^^^^^^^^^^^^^^
> >     'latex_preamble':'\input{my_preamble}',
> >     'documentoptions':'letterpaper,11pt'
> > }
>
> > but self.bibtex seems to think it has a null string.  So once again, what
> > am I doing wrong?
>
> using the key 'use_bibtex' two times, the second time with an empty string.
>
> Günter
>
>
>
> ------------------------------------------------------------------------------
> AlienVault Unified Security Management (USM) platform delivers complete
> security visibility with the essential security capabilities. Easily and
> efficiently configure, manage, and operate all of your security controls
> from a single console and one unified framework. Download a free trial.
> http://p.sf.net/sfu/alienvault_d2d
> _______________________________________________
> Docutils-users mailing list
> Doc...@li...
> https://lists.sourceforge.net/lists/listinfo/docutils-users
>
> Please use "Reply All" to reply to the list.
>



-- 
Michael G. Prisant,
 <Mic...@gm...>

On 2013-05-13, Michael Prisant wrote:

...

> But it looks like the writer is evaluating self.bibtex as null.

null? bool(self.bibtex) == False, because self.bibtex == '' # empty string

> I thought that I was setting this correctly by setting: (all the other
> settings seem to push through correctly to the latex writer with
> appropriate changes when set to different values.

> ltx_overrides = {
>     'input_encoding': 'utf8',
>     'output_encoding': 'utf8',
>     'doctitle_xform': True,
>     'initial_header_level': 2,
>     'use-latex_abstract': True,
>     'use_latex_citations': True,
>     'use_bibtex': 'no_abstract,Articles,Books,Theses',
>     'template': 'my_default.tex',
>     'use_bibtex': '',
      ^^^^^^^^^^^^^^^^^
>     'latex_preamble':'\input{my_preamble}',
>     'documentoptions':'letterpaper,11pt'
> }

> but self.bibtex seems to think it has a null string.  So once again, what
> am I doing wrong?

using the key 'use_bibtex' two times, the second time with an empty string.

Günter

Hi,

Thanks for this response. I posted to this thread asking the maintainers to
provide a simple example of defining a custom role and writing different
versions to html and latex. This was essentially requesting a "toy" version
of how a stock directive was coded: Intercept a custom directive in a
python reading of  rst document tree and be able to publish from that
amended tree to html and latex.

This seems clear enough to me and in my view this question has come up
before though in other forms on the list. The rest of my response follows
in text and should be brief enough for even the extremely time challenged :)

On Tue, May 14, 2013 at 12:30 PM, David Goodger <go...@py...> wrote:

> On Tue, May 14, 2013 at 1:56 AM, Michael Prisant
> <mic...@gm...> wrote:
> > Hi David,
> >
> > Thanks for this reply and for writing docutils. I will try to respond to
> > your questions (to help you help me),  But bear with a me bit -- I don't
> > think that I can directly comply with your comments other than to note
> that
> > "yes I think that the rendered text and functionality" are different in
> the
> > html and latex published versions so it really has to change depending on
> > the output format.
>
> In what way, exactly, do they have to differ? And please: *briefly*.
> Sorry, I don't have time to read long email messages like this one.
>
> The html has adds a bunch of form code; the latex version suppresses
printing various url links etc


> Can your desired output be constructed from primitives that are
> already supported in Docutils? If "yes", your job will be much easier.
> Try. If "no", you may have to derive custom Writers for your
> application (recommended), or go the "raw" directive route you've been
> using (not recommended). But maybe the answer is that Docutils isn't
> the right tool for the job.
>

No for the html output; yes for stuff which is later latex'd.  Once again I
said this in my previous post.

>
> That's about all the advice I can give. I don't know
> bibtex/bibliography details, and I have no desire to learn them.


I don't ask for help with LaTeX/BiBTeX -- just with python
parsing/processing restructured text using your docutils software.[1]


> I
> tried to read your message but found it difficult to follow. I don't
> know what you're trying to accomplish. If you try again, use a
> different approach. Perhaps "show us, don't tell us" will work better.
>
>
I gave a simple version -- but you called it "unnaturally simple", and
asked for a more detailed explanation.  So I di my best (and spent time)
trying to constructively engage the issues which you raised. (Even though I
thought that those issues had been clearly described at least twice before
in preceding posts.) If you read my previous note you will find (i) my reST
input with custom directive, (ii) the html produced from it which I pass
for rendering, and  (iii) the reST which I produce for LaTeXing.  [2]

> First in overview. I am trying to write a sort of annotated bibliography
> > wiki application using reST as the underlying markup language for the
> wiki
> > pages.
>
> ISTM that you may be trying to load too much functionality onto
> Docutils/reST. If you have an interactive web app, Docutils is not the
> right tool for the job. Docutils can transform reST statically, but it
> does not support interactive/dynamic functionality. That's what a web
> framework is for.
>

See above or read my posting -- I think it is clear that that I am *not*
trying to load too much functionality onto Docutils/reST.[3]  I just wanted
help in understanding how to capture the string content my custom directive
by using the docutils tree instead of stock python string processing.
(Actually my "unnaturally simple example" didn't even require capturing the
string contents -- just emitting a different string to the html and the
latex wrtiers when the custom directive was encountered)


> You can still use reST for your text content, but don't try to get
> Docutils to support the database-backed dynamic web stuff.
>
>
It should be clear I am not trying to get you to do this.[4,5] Please read
my posting(s) or at least its(their) first paragraph(s).

Docutils is a big and important project -- chosen by Python to support its
documentation processing. It was arguably brilliantly conceived and coded
and is currently maintained by very gifted software engineers and python
programmers. Users come to it because they think it is better than the
alternatives.

That is why it merits reasonably readable and complete documentation.[6]
And when the user mailing list identifies gaps in that documentation,
developer and maintainers should provide civil responses to their users.[7]
The fact that this sort of docutils directive "hacking" question (albeit in
different forms) has come up repeatedly on this list and elsewhere  on the
web should be persuasive enough argument for the developer and maintainers
that more documentation and examples beyond the source code are needed --
if only to save themselves time in responding. But ultimately that is their
choice.

Michael

[1] I already read your note to Allan Issac on this list in which you
expressed similar sentiments: That is why I didn't incorporate any
LaTeX/BiBTeX code in my postings.

[2] I appreciate that the note was long and you are busy. It may surprise
you that this is also true for your docutils users whom you ask to jump
through hoops and whose time is also burnt up when posting. All of us
participate -- post and respond -- on this list by choice. But please don't
blow off answering my questions by demanding endless clarifications and
then incorrectly claiming that I haven't provided what you asked for.

[3] How do I know this? I have implemented a working solution but it uses
stock python (clearly kludgey but < 100 lines) to parse the reST source
(all it does is capture the custom role contents which is really just a
string and replace it with another string which provides the expanded
content based on the list of keys).

[4] In no small measure because I have read your multiple previous posting
on this list responding to other users saying you won't go in that
direction of providing any database/citation/bibtex support in docutils.
But also note [5]

[5] In fact Guenter and Englebert have already provided the reST citation
directive with potential bibtex functionality when reST is translated into
LaTeX  See the use-bibtex option in rst2latex (in the current distribution
of python-docutils) and rst2pdf.py -- authored by Guenter and on the web --
and corresponding code.  But at least in this posting thread I haven't been
using this route nor asking about it.

[6] For example "The Docutils Hacker's Guide" has been left incomplete for
years. Completing this would take work for the developers but might be "the
stitch in time that saves nine"  It certainly IMHO would cut down on
aggravation in the docutils-user list. Again your time, your project, your
choice.

[7] It should be obvious that users are actually project supporters and not
developer/maintainer adversaries. They don't deserve to be essentially told
to get lost -- eg Maybe "Docutils isn't the right tool for the job" -- if
the code author/maintainer doesn't like/understand their questions or the
issues being raised.

 -- David Goodger
>
> >  In my view it provided a richer markup set than markdown and more
> > natural and pythonic code base for extension. Unwinding my actual code
> is a
> > little bit difficult because it is interwined with the python web
> framework
> > "BottlePy" that I am using and its native simple template.  But I will
> try
> > to provide more explanation to better help you help me.
> >
> > It seems to me what I want to do is "publish" html and latex from the
> same
> > reST input source and by using the doctree nodes  Actually the modality
> of
> > producing the source to be published by the latex writer is along the
> lines
> > that I think that you suggested ie producing transformed reST for further
> > processing  But the translation to html is somewhat different because I
> am
> > want to produce an interactive web page.
> >
> > Now in detail. I have been trying to insert annotated bibliographies
> within
> > my reST documents using a custom "bibentries" directive.
> >
> > A more real example piece of my typical reST source would look like this:
> >
> > .. bibentries::
> >
> >     Bad85 This is a note for a single paper; the markup end delimiter
> >     is two semicolons allowing a single semicolon in the note;;
> >
> >     PB94;;
> >
> >     Dwy04;;
> >
> > The custom role directive wraps a list of entries. A given document might
> > have 4-5 of these lists. The entries are delimited by two ";;" and can
> be on
> > the same or separate lines.  Each entry has two subfields separated by a
> > space:  the first subfield is a citation key which identifies the
> reference
> > for my bibtex databases and the second is a note to be attached to the
> > paper.
> >
> > Once this is gathered I can much it on my own  -- the directive snippet
> that
> > I proved to be an exceptionally easy way of scooping up the bib entry
> data
> > from the reST document.
> >
> > I then produce somewhat more complex html: the entries become a list. The
> > rendered page provides some form interaction options for the user.  This
> > requires a translation to "fatten" the reST source. Here is a snippet of
> > html that I produce via the "raw" approach using the custom directive
> coded
> > in a very similar fashion to my note.  This seems to run counter to the
> > advice
> >
> > <ol>
> >
> > <li>
> > <!-- -->
> > <input type="checkbox" name="entries" value="Articles/Bad85"
> > class="chkAll"/>
> > <!-- -->
> > [Articles/<a href = "/Bibs/Articles/Bad85">Bad85</a>:
> > <a href = "http://pubs.acs.org/doi/abs/10.1021/ar00109a003">url</a>
> > <a href="/https/sourceforge.net/data/Pdfs/Articles/Bad85.pdf">pdf</a>]
> > Bader.
> > Atoms in molecules.
> > In <i>Accounts of Chemical Research</i>
> > <b>18</b> (1): p. 9-15, 1985
> > <blockquote class="abstract">
> > Approximate quantum mechanical state functions have recently been
> > obtained for the norbornyl cation. These calculations have yielded the
> > energy of its equilibrium geometry and the relative energies of
> > neighboring geometries. It is the purpose of this account to
> > illustrate that more chemical information than simply energies and
> > their associated geometries can be obtained directly from a state
> > function. A state function contains the necessary information to both
> > define the atoms in a molecule and determine their average properties.
> > It also enables one to assign a structure, that is, determine the
> > network of bonds linking the atoms in a molecule and determine whether
> > or not the structure is table. The state function further determines
> > where electronic charge is locally concentrated and depleted. Quantum
> > mechanics can be used to relate these properties to local energy
> > contributions, thereby providing an understanding of the geometry and
> > reactivity of a molecule. (Summary prepared by MGP 120530)
> > </blockquote>
> > <p>Note: This is a note for a single paper; the markup end delimiter is
> two
> > semicolons allowing a single semicolon in the note</p>
> > </li>
> >
> >
> > This part worked fine for me.  But now I want to translate the page to
> latex
> > both for pdf printing and possible collaborative documents.  I like the
> way
> > the doctree scoops up the custom directive contents.  And in this case I
> > just retranslate my original reST source to a new reST document with the
> > bibliographic information for latexing -- actually easier. But I don't
> > understand the right "Docutilic" idiom for coding this using the parsed
> > doctree and then "publishing" from the same reST source to either html or
> > latex as needed.
> >
> > So right now I am just munching on the original reST source to extract
> the
> > bibentries stuff, use it to create my annotated bibliography list, and
> then
> > write it a reST document for latexing. (This part is also working) Here
> is a
> > snippet of my retranslated resT which can be published to latex with the
> > latex2e writer (and processed to pdf):
> >
> > #.
> >     [Articles/Bad85:
> >     url
> >     pdf]
> >     Bader.
> >     Atoms in molecules.
> >     In *Accounts of Chemical Research*
> >     **18**
> >     (1):
> >     p. 9-15,
> >     1985
> >
> >          Approximate quantum mechanical state functions have recently
> >          been obtained for the norbornyl cation. These calculations
> >          have yielded the energy of its equilibrium geometry and the
> >          relative energies of neighboring geometries. It is the
> >          purpose of this account to illustrate that more chemical
> >          information than simply energies and their associated
> >          geometries can be obtained directly from a state function. A
> >          state function contains the necessary information to both
> >          define the atoms in a molecule and determine their average
> >          properties. It also enables one to assign a structure, that
> >          is, determine the network of bonds linking the atoms in a
> >          molecule and determine whether or not the structure is table.
> >          The state function further determines where electronic charge
> >          is locally concentrated and depleted. Quantum mechanics can
> >          be used to relate these properties to local energy
> >          contributions, thereby providing an understanding of the
> >          geometry and reactivity of a molecule. (Summary prepared by
> >          MGP 120530)
> >
> >     Note: This is a note for a single paper; the markup end delimiter is
> two
> > semicolons allowing a single semicolon in the note
> >
> >
> > Once again thanks to you, the maintainers, and the list  for following
> > through on reading my somewhat extended pposts. Docutils is a big
> project of
> > immense and of self-evident value to the Python community and beyond.  I
> > understand that you and your colleagues are volunteering your valuable to
> > time to support this project.
> >
> > Michael
> >
> >
> >
> > On Tue, May 14, 2013 at 12:29 AM, David Goodger <go...@py...>
> wrote:
> >>
> >> On Mon, May 13, 2013 at 8:13 PM, Michael Prisant
> >> <mic...@gm...> wrote:
> >> > Yes this is a dumb example but I think it is enough to get started.
> How
> >> > can
> >> > this question be made more simple or specific?
> >>
> >> You can start by giving us a minimal but *real* example of what you
> >> want as output. I think you're over-complicating the issue by trying
> >> to simplify it in an unnatural way.
> >>
> >> Unnatural, because the it's the job of the Writer classes to translate
> >> Docutils document trees (doctrees) into their final formats. Don't try
> >> to do the job of Writers from within your directive, that's the wrong
> >> approach. You should be trying to write correct doctree nodes from
> >> your code.
> >>
> >> Apart from the differences in markup/tagging/codes, do you really need
> >> different output from the two writers? IOW, does the content itself
> >> (the rendered text & functionality) have to change depending on the
> >> output format?
> >>
> >> Don't worry about the HTML or LaTeX output, except to illustrate what
> >> you want. Show us some real input, and some real desired output.
> >>
> >> > PS Hoping that this can be posted to list in under 5 days!
> >>
> >> Are you talking about your message, or the reply? If your message, it
> >> could be that the first time you posted, you weren't a member, and
> >> somebody (me) had to approve the post first, and I was otherwise busy.
> >> Sometimes it takes me a while to get to list moderation.
> >>
> >> If you're talking about the reply, well, sorry, but sometimes it takes
> >> a while for people to carve time out of their busy lives.
> >>
> >> --
> >> David Goodger <http://python.net/~goodger>
> >
> >
> >
> >
> > --
> > Michael G. Prisant
>



-- 
Michael G. Prisant
 <Mic...@gm...>

On Tue, May 14, 2013 at 1:56 AM, Michael Prisant
<mic...@gm...> wrote:
> Hi David,
>
> Thanks for this reply and for writing docutils. I will try to respond to
> your questions (to help you help me),  But bear with a me bit -- I don't
> think that I can directly comply with your comments other than to note that
> "yes I think that the rendered text and functionality" are different in the
> html and latex published versions so it really has to change depending on
> the output format.

In what way, exactly, do they have to differ? And please: *briefly*.
Sorry, I don't have time to read long email messages like this one.

Can your desired output be constructed from primitives that are
already supported in Docutils? If "yes", your job will be much easier.
Try. If "no", you may have to derive custom Writers for your
application (recommended), or go the "raw" directive route you've been
using (not recommended). But maybe the answer is that Docutils isn't
the right tool for the job.

That's about all the advice I can give. I don't know
bibtex/bibliography details, and I have no desire to learn them. I
tried to read your message but found it difficult to follow. I don't
know what you're trying to accomplish. If you try again, use a
different approach. Perhaps "show us, don't tell us" will work better.

> First in overview. I am trying to write a sort of annotated bibliography
> wiki application using reST as the underlying markup language for the wiki
> pages.

ISTM that you may be trying to load too much functionality onto
Docutils/reST. If you have an interactive web app, Docutils is not the
right tool for the job. Docutils can transform reST statically, but it
does not support interactive/dynamic functionality. That's what a web
framework is for.

You can still use reST for your text content, but don't try to get
Docutils to support the database-backed dynamic web stuff.

-- David Goodger

>  In my view it provided a richer markup set than markdown and more
> natural and pythonic code base for extension. Unwinding my actual code is a
> little bit difficult because it is interwined with the python web framework
> "BottlePy" that I am using and its native simple template.  But I will try
> to provide more explanation to better help you help me.
>
> It seems to me what I want to do is "publish" html and latex from the same
> reST input source and by using the doctree nodes  Actually the modality of
> producing the source to be published by the latex writer is along the lines
> that I think that you suggested ie producing transformed reST for further
> processing  But the translation to html is somewhat different because I am
> want to produce an interactive web page.
>
> Now in detail. I have been trying to insert annotated bibliographies within
> my reST documents using a custom "bibentries" directive.
>
> A more real example piece of my typical reST source would look like this:
>
> .. bibentries::
>
>     Bad85 This is a note for a single paper; the markup end delimiter
>     is two semicolons allowing a single semicolon in the note;;
>
>     PB94;;
>
>     Dwy04;;
>
> The custom role directive wraps a list of entries. A given document might
> have 4-5 of these lists. The entries are delimited by two ";;" and can be on
> the same or separate lines.  Each entry has two subfields separated by a
> space:  the first subfield is a citation key which identifies the reference
> for my bibtex databases and the second is a note to be attached to the
> paper.
>
> Once this is gathered I can much it on my own  -- the directive snippet that
> I proved to be an exceptionally easy way of scooping up the bib entry data
> from the reST document.
>
> I then produce somewhat more complex html: the entries become a list. The
> rendered page provides some form interaction options for the user.  This
> requires a translation to "fatten" the reST source. Here is a snippet of
> html that I produce via the "raw" approach using the custom directive coded
> in a very similar fashion to my note.  This seems to run counter to the
> advice
>
> <ol>
>
> <li>
> <!-- -->
> <input type="checkbox" name="entries" value="Articles/Bad85"
> class="chkAll"/>
> <!-- -->
> [Articles/<a href = "/Bibs/Articles/Bad85">Bad85</a>:
> <a href = "http://pubs.acs.org/doi/abs/10.1021/ar00109a003">url</a>
> <a href="/https/sourceforge.net/data/Pdfs/Articles/Bad85.pdf">pdf</a>]
> Bader.
> Atoms in molecules.
> In <i>Accounts of Chemical Research</i>
> <b>18</b> (1): p. 9-15, 1985
> <blockquote class="abstract">
> Approximate quantum mechanical state functions have recently been
> obtained for the norbornyl cation. These calculations have yielded the
> energy of its equilibrium geometry and the relative energies of
> neighboring geometries. It is the purpose of this account to
> illustrate that more chemical information than simply energies and
> their associated geometries can be obtained directly from a state
> function. A state function contains the necessary information to both
> define the atoms in a molecule and determine their average properties.
> It also enables one to assign a structure, that is, determine the
> network of bonds linking the atoms in a molecule and determine whether
> or not the structure is table. The state function further determines
> where electronic charge is locally concentrated and depleted. Quantum
> mechanics can be used to relate these properties to local energy
> contributions, thereby providing an understanding of the geometry and
> reactivity of a molecule. (Summary prepared by MGP 120530)
> </blockquote>
> <p>Note: This is a note for a single paper; the markup end delimiter is two
> semicolons allowing a single semicolon in the note</p>
> </li>
>
>
> This part worked fine for me.  But now I want to translate the page to latex
> both for pdf printing and possible collaborative documents.  I like the way
> the doctree scoops up the custom directive contents.  And in this case I
> just retranslate my original reST source to a new reST document with the
> bibliographic information for latexing -- actually easier. But I don't
> understand the right "Docutilic" idiom for coding this using the parsed
> doctree and then "publishing" from the same reST source to either html or
> latex as needed.
>
> So right now I am just munching on the original reST source to extract the
> bibentries stuff, use it to create my annotated bibliography list, and then
> write it a reST document for latexing. (This part is also working) Here is a
> snippet of my retranslated resT which can be published to latex with the
> latex2e writer (and processed to pdf):
>
> #.
>     [Articles/Bad85:
>     url
>     pdf]
>     Bader.
>     Atoms in molecules.
>     In *Accounts of Chemical Research*
>     **18**
>     (1):
>     p. 9-15,
>     1985
>
>          Approximate quantum mechanical state functions have recently
>          been obtained for the norbornyl cation. These calculations
>          have yielded the energy of its equilibrium geometry and the
>          relative energies of neighboring geometries. It is the
>          purpose of this account to illustrate that more chemical
>          information than simply energies and their associated
>          geometries can be obtained directly from a state function. A
>          state function contains the necessary information to both
>          define the atoms in a molecule and determine their average
>          properties. It also enables one to assign a structure, that
>          is, determine the network of bonds linking the atoms in a
>          molecule and determine whether or not the structure is table.
>          The state function further determines where electronic charge
>          is locally concentrated and depleted. Quantum mechanics can
>          be used to relate these properties to local energy
>          contributions, thereby providing an understanding of the
>          geometry and reactivity of a molecule. (Summary prepared by
>          MGP 120530)
>
>     Note: This is a note for a single paper; the markup end delimiter is two
> semicolons allowing a single semicolon in the note
>
>
> Once again thanks to you, the maintainers, and the list  for following
> through on reading my somewhat extended pposts. Docutils is a big project of
> immense and of self-evident value to the Python community and beyond.  I
> understand that you and your colleagues are volunteering your valuable to
> time to support this project.
>
> Michael
>
>
>
> On Tue, May 14, 2013 at 12:29 AM, David Goodger <go...@py...> wrote:
>>
>> On Mon, May 13, 2013 at 8:13 PM, Michael Prisant
>> <mic...@gm...> wrote:
>> > Yes this is a dumb example but I think it is enough to get started. How
>> > can
>> > this question be made more simple or specific?
>>
>> You can start by giving us a minimal but *real* example of what you
>> want as output. I think you're over-complicating the issue by trying
>> to simplify it in an unnatural way.
>>
>> Unnatural, because the it's the job of the Writer classes to translate
>> Docutils document trees (doctrees) into their final formats. Don't try
>> to do the job of Writers from within your directive, that's the wrong
>> approach. You should be trying to write correct doctree nodes from
>> your code.
>>
>> Apart from the differences in markup/tagging/codes, do you really need
>> different output from the two writers? IOW, does the content itself
>> (the rendered text & functionality) have to change depending on the
>> output format?
>>
>> Don't worry about the HTML or LaTeX output, except to illustrate what
>> you want. Show us some real input, and some real desired output.
>>
>> > PS Hoping that this can be posted to list in under 5 days!
>>
>> Are you talking about your message, or the reply? If your message, it
>> could be that the first time you posted, you weren't a member, and
>> somebody (me) had to approve the post first, and I was otherwise busy.
>> Sometimes it takes me a while to get to list moderation.
>>
>> If you're talking about the reply, well, sorry, but sometimes it takes
>> a while for people to carve time out of their busy lives.
>>
>> --
>> David Goodger <http://python.net/~goodger>
>
>
>
>
> --
> Michael G. Prisant

Hi David,

Thanks for this reply and for writing docutils. I will try to respond to
your questions (to help you help me),  But bear with a me bit -- I don't
think that I can directly comply with your comments other than to note that
"yes I think that the rendered text and functionality" are different in the
html and latex published versions so it really has to change depending on
the output format.

First in overview. I am trying to write a sort of annotated bibliography
wiki application using reST as the underlying markup language for the wiki
pages.  In my view it provided a richer markup set than markdown and more
natural and pythonic code base for extension. Unwinding my actual code is a
little bit difficult because it is interwined with the python web framework
"BottlePy" that I am using and its native simple template.  But I will try
to provide more explanation to better help you help me.

It seems to me what I want to do is "publish" html and latex from the same
reST input source and by using the doctree nodes  Actually the modality of
producing the source to be published by the latex writer is along the lines
that I think that you suggested ie producing transformed reST for further
processing  But the translation to html is somewhat different because I am
want to produce an interactive web page.

Now in detail. I have been trying to insert annotated bibliographies within
my reST documents using a custom "bibentries" directive.

A more real example piece of my typical reST source would look like this:

.. bibentries::
    Bad85 This is a note for a single paper; the markup end delimiter
    is two semicolons allowing a single semicolon in the note;;

    PB94;;

    Dwy04;;

The custom role directive wraps a list of entries. A given document might
have 4-5 of these lists. The entries are delimited by two ";;" and can be
on the same or separate lines.  Each entry has two subfields separated by a
space:  the first subfield is a citation key which identifies the reference
for my bibtex databases and the second is a note to be attached to the
paper.

Once this is gathered I can much it on my own  -- the directive snippet
that I proved to be an exceptionally easy way of scooping up the bib entry
data from the reST document.

I then produce somewhat more complex html: the entries become a list. The
rendered page provides some form interaction options for the user.
This requires
a translation to "fatten" the reST source. Here is a snippet of html that I
produce via the "raw" approach using the custom directive coded in a very
similar fashion to my note.  This seems to run counter to the advice

<ol><li><!-- --><input type="checkbox" name="entries"
value="Articles/Bad85" class="chkAll"/><!-- -->[Articles/<a href =
"/Bibs/Articles/Bad85
<view-source:http://localhost:8090/Bibs/Articles/Bad85>">Bad85</a>:<a
href = "http://pubs.acs.org/doi/abs/10.1021/ar00109a003
<view-source:http://pubs.acs.org/doi/abs/10.1021/ar00109a003>">url</a><a
href="/https/sourceforge.net/data/Pdfs/Articles/Bad85.pdf
<view-source:http://localhost:8090/data/Pdfs/Articles/Bad85.pdf>">pdf</a>]Bader.Atoms
in molecules.In <i>Accounts of Chemical Research</i><b>18</b> (1): p.
9-15, 1985<blockquote class="abstract">Approximate quantum mechanical
state functions have recently beenobtained for the norbornyl cation.
These calculations have yielded theenergy of its equilibrium geometry
and the relative energies ofneighboring geometries. It is the purpose
of this account toillustrate that more chemical information than
simply energies andtheir associated geometries can be obtained
directly from a statefunction. A state function contains the necessary
information to bothdefine the atoms in a molecule and determine their
average properties.It also enables one to assign a structure, that is,
determine thenetwork of bonds linking the atoms in a molecule and
determine whetheror not the structure is table. The state function
further determineswhere electronic charge is locally concentrated and
depleted. Quantummechanics can be used to relate these properties to
local energycontributions, thereby providing an understanding of the
geometry andreactivity of a molecule. (Summary prepared by MGP
120530)</blockquote><p>Note: This is a note for a single paper; the
markup end delimiter is two semicolons allowing a single semicolon in
the note</p></li>


This part worked fine for me.  But now I want to translate the page to
latex both for pdf printing and possible collaborative documents.  I like
the way the doctree scoops up the custom directive contents.  And in this
case I just retranslate my original reST source to a new reST document with
the bibliographic information for latexing -- actually easier. But I don't
understand the right "Docutilic" idiom for coding this using the parsed
doctree and then "publishing" from the same reST source to either html or
latex as needed.

So right now I am just munching on the original reST source to extract the
bibentries stuff, use it to create my annotated bibliography list, and then
write it a reST document for latexing. (This part is also working) Here is
a snippet of my retranslated resT which can be published to latex with the
latex2e writer (and processed to pdf):

#.
    [Articles/Bad85:
    url
    pdf]
    Bader.
    Atoms in molecules.
    In *Accounts of Chemical Research*
    **18**
    (1):
    p. 9-15,
    1985

         Approximate quantum mechanical state functions have recently
         been obtained for the norbornyl cation. These calculations
         have yielded the energy of its equilibrium geometry and the
         relative energies of neighboring geometries. It is the
         purpose of this account to illustrate that more chemical
         information than simply energies and their associated
         geometries can be obtained directly from a state function. A
         state function contains the necessary information to both
         define the atoms in a molecule and determine their average
         properties. It also enables one to assign a structure, that
         is, determine the network of bonds linking the atoms in a
         molecule and determine whether or not the structure is table.
         The state function further determines where electronic charge
         is locally concentrated and depleted. Quantum mechanics can
         be used to relate these properties to local energy
         contributions, thereby providing an understanding of the
         geometry and reactivity of a molecule. (Summary prepared by
         MGP 120530)

    Note: This is a note for a single paper; the markup end delimiter
is two semicolons allowing a single semicolon in the note


Once again thanks to you, the maintainers, and the list  for following
through on reading my somewhat extended pposts. Docutils is a big project
of immense and of self-evident value to the Python community and
beyond.  I understand
that you and your colleagues are volunteering your valuable to time to
support this project.

Michael



On Tue, May 14, 2013 at 12:29 AM, David Goodger <go...@py...> wrote:

> On Mon, May 13, 2013 at 8:13 PM, Michael Prisant
> <mic...@gm...> wrote:
> > Yes this is a dumb example but I think it is enough to get started. How
> can
> > this question be made more simple or specific?
>
> You can start by giving us a minimal but *real* example of what you
> want as output. I think you're over-complicating the issue by trying
> to simplify it in an unnatural way.
>
> Unnatural, because the it's the job of the Writer classes to translate
> Docutils document trees (doctrees) into their final formats. Don't try
> to do the job of Writers from within your directive, that's the wrong
> approach. You should be trying to write correct doctree nodes from
> your code.
>
> Apart from the differences in markup/tagging/codes, do you really need
> different output from the two writers? IOW, does the content itself
> (the rendered text & functionality) have to change depending on the
> output format?
>
> Don't worry about the HTML or LaTeX output, except to illustrate what
> you want. Show us some real input, and some real desired output.
>
> > PS Hoping that this can be posted to list in under 5 days!
>
> Are you talking about your message, or the reply? If your message, it
> could be that the first time you posted, you weren't a member, and
> somebody (me) had to approve the post first, and I was otherwise busy.
> Sometimes it takes me a while to get to list moderation.
>
> If you're talking about the reply, well, sorry, but sometimes it takes
> a while for people to carve time out of their busy lives.
>
> --
> David Goodger <http://python.net/~goodger>
>



-- 
Michael G. Prisant
 <Mic...@gm...>

On Mon, May 13, 2013 at 8:13 PM, Michael Prisant
<mic...@gm...> wrote:
> Yes this is a dumb example but I think it is enough to get started. How can
> this question be made more simple or specific?

You can start by giving us a minimal but *real* example of what you
want as output. I think you're over-complicating the issue by trying
to simplify it in an unnatural way.

Unnatural, because the it's the job of the Writer classes to translate
Docutils document trees (doctrees) into their final formats. Don't try
to do the job of Writers from within your directive, that's the wrong
approach. You should be trying to write correct doctree nodes from
your code.

Apart from the differences in markup/tagging/codes, do you really need
different output from the two writers? IOW, does the content itself
(the rendered text & functionality) have to change depending on the
output format?

Don't worry about the HTML or LaTeX output, except to illustrate what
you want. Show us some real input, and some real desired output.

> PS Hoping that this can be posted to list in under 5 days!

Are you talking about your message, or the reply? If your message, it
could be that the first time you posted, you weren't a member, and
somebody (me) had to approve the post first, and I was otherwise busy.
Sometimes it takes me a while to get to list moderation.

If you're talking about the reply, well, sorry, but sometimes it takes
a while for people to carve time out of their busy lives.

--
David Goodger <http://python.net/~goodger>

Hi

OK after reading Guenter's note I see my misunderstanding of this piece of
code:

return [nodes.raw('', biblsthtml, format='html')]

but my question still stands.

As noted I had adapted it from another source; it seemed to do what I
wanted. But I didn't really understand what it did. This mistake may have
created some confusion around my underlying request for explanation and
sample code.

>From Guenter's reponse I now understand that this was more or less
equivalent to using the raw directive in the reST source. Which in that
manner -- ie raw directives in the rest source -- as Guenter noted could be
adapted and targeted to the various other writers.

But my specific question remains.  If I have the custom directive

.. bibentries::

    BCC+87;;

How do I write python code in the proper "Docutilic idiom" perhaps
resembling my start with

class BibDirective(Directive):
    has_content = True
    def run(self):
...
directives.register_directive("bibentries", BibDirective)

which will allow me to translate the content to both html and latex.

So for purposes of explanation lets try for '<pre> This is the html
bibdirective </pre>' to be emitted for html when I call

html_string = publish_string(\
        source=content,\
        writer_name='html',\
        settings_overrides=htm_overrides)

and '\begin{verbatim} This is the latex bibdirective \end{verbatim}' to be
emitted when I call:

latex_string = publish_string(\
            source=content,\
            writer_name='latex2e',\
            settings_overrides=ltx_overrides)

Yes this is a dumb example but I think it is enough to get started. How can
this question be made more simple or specific? Guenter is right in pointing
out that the coding task is resembles coding the stock directives in the
docutils source and in principle could begin there.

But there is a lot of complexity in the docutils code base that is not
relevant to my (or other "hacker/user) needs. That complexity makes it hard
-- at least for me -- to use the docutils source as a basis for
understanding how to "hack" extensions for personal use. In any case the
starting point for my question is that I haven't been able to figure out
how on my own to adapt the code in the docutils source for what I view as a
modest extensions. (Donc J'affiche a la liste apres avoir passe quelques
semaines a la recherche de cette question)

Responding with a simplified "toy" python example which would be my first
preference.  That response might also be the basis for an "examples"
addendum to the incomplete "Docutils Hackers Guide" And I not sure that I
understand why there is so much resistance to just answering this
question.

If providing a "toy example" is not viewed as a legitimate request or is
too burdensome, perhaps the list could at least identify specific pieces of
code or one specific directive in the docutils source which would be most
relevant for my needs and "easiest" to hack rather than just sending me
back to the source in general.

Michael

PS Hoping that this can be posted to list in under 5 days!


On Mon, May 13, 2013 at 3:51 PM, Guenter Milde <mi...@us...> wrote:

> On 2013-05-13, Michael Prisant wrote:
>
> > Thanks for this reply but I was looking for more specific advice with
> > regard to coding rest custom directives (as opposed to using the "raw:
> > approach) which are properly translated to both html and latex (like
> > non-custom directives).
>
> For specific adwise, we need a specific task. Otherwise, I could just
> copy one of the standard directive classes (all of them are good examples
> and (almost) all or them are properly translated to all output formats
> that Docutils supports.
>
> > Admittedly, I tried to pare my actual python code snippet to a bare
> > minimum more or else working example. (This eliminated my actual bibtex
> > processing which is what actually necessitated python code as opposed
> > to just using "raw".  So some confusion seems to stem from the example
> > given which doesn't do very much) On this basis I was hoping that one
> > of the experienced coders could offer an *explicit* but *simple* python
> > example to the group based on the very *simple* code snippets that I
> > provided.
>
> The problem I have with your example is, that it uses "raw" without any
> need. I don't know for certain, but I don't think "raw" is required for
> the real task either. Therefore, a solution to the stated problem of
> providing two raw nodes -- one for latex one for html -- is just going
> further in the wrong direction, away from a generic solution to the real
> problem behind the example.
>
> ...
>
> > The question here is really to help users who are attempting to code
> simple
> > extensions to reST for their own specific uses as opposed to general
> > development which would be folded into the software.  I want to
> translate a
> > custom reST directive into just html and latex (as opposed to also
> > translating into every possible output format) [2]
>
> Did you already try to let the directive class return a list of "raw"
> nodes (one for each supported output format)? AFAIK, "raw" nodes of the
> "wrong" format are just ignored by the writers. Caveat: I did not test
> this approach nor do I know whether it works this way.
>
> Günter
>
>
>
> ------------------------------------------------------------------------------
> AlienVault Unified Security Management (USM) platform delivers complete
> security visibility with the essential security capabilities. Easily and
> efficiently configure, manage, and operate all of your security controls
> from a single console and one unified framework. Download a free trial.
> http://p.sf.net/sfu/alienvault_d2d
> _______________________________________________
> Docutils-users mailing list
> Doc...@li...
> https://lists.sourceforge.net/lists/listinfo/docutils-users
>
> Please use "Reply All" to reply to the list.
>



-- 
Michael G. Prisant,


 <Mic...@gm...>

Hi,

Answering my own question but posing a followup.  I am getting a "double"
references section because I substitute a custom template on translation to
latex which contains the following code adding a bibliography:

$head_prefix% generated by Docutils <http://docutils.sourceforge.net/>
\usepackage{fixltx2e} % LaTeX patches, \textsubscript
\usepackage{cmap} % fix search and cut-and-paste in Acrobat
$requirements
%%% Custom LaTeX preamble
...
\bibliography{Articles,Books,Theses}
\bibliographystyle{no_abstract}
\end{body}

Eliminating the problematic stuff from my template it would seem that the
relevant code in  docutils/writers/latex2e/__init__.py should do the right
thing if the use_bibtex string is not null

        # * bibliography
        #   TODO insertion point of bibliography should be configurable.
        if self._use_latex_citations and len(self._bibitems)>0:
            if not self.bibtex:
                widest_label = ''
                for bi in self._bibitems:
                    if len(widest_label)<len(bi[0]):
                        widest_label = bi[0]
                self.out.append('\n\\begin{thebibliography}{%s}\n' %
                                 widest_label)
                for bi in self._bibitems:
                    # cite_key: underscores must not be escaped
                    cite_key = bi[0].replace(r'\_','_')
                    self.out.append('\\bibitem[%s]{%s}{%s}\n' %
                                     (bi[0], cite_key, bi[1]))
                self.out.append('\\end{thebibliography}\n')
            else:
                self.out.append('\n\\bibliographystyle{%s}\n' %
                                self.bibtex[0])
                self.out.append('\\bibliography{%s}\n' % self.bibtex[1])

But it looks like the writer is evaluating self.bibtex as null.

I thought that I was setting this correctly by setting: (all the other
settings seem to push through correctly to the latex writer with
appropriate changes when set to different values.

ltx_overrides = {
    'input_encoding': 'utf8',
    'output_encoding': 'utf8',
    'doctitle_xform': True,
    'initial_header_level': 2,
    'use-latex_abstract': True,
    'use_latex_citations': True,
    'use_bibtex': 'no_abstract,Articles,Books,Theses',
    'template': 'my_default.tex',
    'use_bibtex': '',
    'latex_preamble':'\input{my_preamble}',
    'documentoptions':'letterpaper,11pt'
}

but self.bibtex seems to think it has a null string.  So once again, what
am I doing wrong?

Michael


On Mon, May 13, 2013 at 12:50 PM, Michael Prisant <mic...@gm...
> wrote:

> Hi,
>
> I have been experimenting with the --use-bibtex=mybibsty,mybib option.
> The good news is I am able to this to properly process citations in reST.
> But the latex translation creates tex code on processing which has two
> references sections.  Is this the way it is supposed to work? What am I
> doing wrong?
>
> Here is my example:
>
> - This reST snippet (extracted and edited):
>
> Here is a citation [HLD+00]_. Here is more text with citations [PM60]_.
> Donec id elit non mi porta gravida at eget metus.
> Fusce dapibus, tellus accursus commodo, tortormauris condimentum nibh, ut
> fermentum massa justo sit ametrisus.
> Etiam porta sem malesuada magna mollis euismod [Dwy04]_.
>
> .. [HLD+00]
> .. [PM60]
> .. [Dwy04]
>
> - Produces this tex (extracted and edited):
>
> Here is a citation \cite{HLD+00}. Here is more text with citations \cite{PM60}.
>
> Donec id elit non mi porta gravida at eget metus.
> Fusce dapibus, tellus ac cursus commodo,tortor mauris condimentum nibh, ut fermentum massa justo sit ametrisus.
> Etiam porta sem malesuada magna mollis euismod \cite{Dwy04}.
>
>
> \begin{thebibliography}{HLD+00}
> \bibitem[HLD+00]{HLD+00}{}
> \bibitem[PM60]{PM60}{}
> \bibitem[Dwy04]{Dwy04}{}
> \end{thebibliography}
>
> \bibliography{Articles,Books,Theses}
> \bibliographystyle{no_abstract}
>
> - This in turn produces two bibliographies when processed to pdf.  I would
> like to get rid of the "\begin{thebibliography}..\end{thebibliography}" without
> kludgey postprocessing of the tex source. Can this be done?
>
> Michael
>
> --
> Michael G. Prisant,
>
>
>  <Mic...@gm...>




-- 
Michael G. Prisant,

 <Mic...@gm...>

On 2013-05-13, Michael Prisant wrote:

> Thanks for this reply but I was looking for more specific advice with
> regard to coding rest custom directives (as opposed to using the "raw:
> approach) which are properly translated to both html and latex (like
> non-custom directives). 

For specific adwise, we need a specific task. Otherwise, I could just
copy one of the standard directive classes (all of them are good examples
and (almost) all or them are properly translated to all output formats
that Docutils supports.

> Admittedly, I tried to pare my actual python code snippet to a bare
> minimum more or else working example. (This eliminated my actual bibtex
> processing which is what actually necessitated python code as opposed
> to just using "raw".  So some confusion seems to stem from the example
> given which doesn't do very much) On this basis I was hoping that one
> of the experienced coders could offer an *explicit* but *simple* python
> example to the group based on the very *simple* code snippets that I
> provided.

The problem I have with your example is, that it uses "raw" without any
need. I don't know for certain, but I don't think "raw" is required for
the real task either. Therefore, a solution to the stated problem of
providing two raw nodes -- one for latex one for html -- is just going
further in the wrong direction, away from a generic solution to the real
problem behind the example. 

...

> The question here is really to help users who are attempting to code simple
> extensions to reST for their own specific uses as opposed to general
> development which would be folded into the software.  I want to translate a
> custom reST directive into just html and latex (as opposed to also
> translating into every possible output format) [2]

Did you already try to let the directive class return a list of "raw"
nodes (one for each supported output format)? AFAIK, "raw" nodes of the
"wrong" format are just ignored by the writers. Caveat: I did not test
this approach nor do I know whether it works this way.

Günter