Revision 641632 of "Benutzer Diskussion:Dirk Huenniger/wb2pdf/ubuntu microtype" on dewikibooks

Hi Dirk,

I did these steps on my Ubuntu machine:
<small><pre>
 sudo apt-get install subversion ghc libghc6-regex-compat-dev libghc6-http-dev python3.1
 sudo apt-get install texlive-latex-extra texlive-fonts-extra librsvg2-bin imagemagick texlive-games
 sudo apt-get install texlive-fonts-recommended texlive-lang-cyrillic texlive-lang-greek arabtex texlive-science
 sudo apt-get install texlive-humanities python3.1-tk texlive-full libghc6-missingh-dev ttf-freefont
 sudo apt-get install libghc6-split-dev cabal-install libghc6-hxt-dev latex-cjk-all cm-super-minimal ttf-unifont
 sudo apt-get install ttf-wqy-zenhei python-fontforge
 sudo cabal update
 sudo cabal install spoon hxt hxt-http url
 svn co https://wb2pdf.svn.sourceforge.net/svnroot/wb2pdf@869 wb2pdf 
 cd wb2pdf/trunk/src
 python3.1 makelinuxbins.py
</pre></small>
And I got from the last step:
<small><pre>
ghc: unrecognised flags: -rtsopts
Usage: For basic information, try the `--help' option.
Traceback (most recent call last):
  File "makelinuxbins.py", line 6, in <module>
    shutil.copy("pa","../bin/pa")
  File "/usr/lib/python3.1/shutil.py", line 99, in copy
    copyfile(src, dst)
  File "/usr/lib/python3.1/shutil.py", line 65, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: 'pa'
</pre></small>
The directory /wb2pdf/trunk/bin is empty except for a folder .svn. Any ideas? - Matthias

It depends on the version of GHC compiler. Try to remove  -rtsopts in makelinuxbins.py [[Benutzer:Dirk Huenniger|Dirk Huenniger]] 13:50, 19. Jan. 2012 (CET)
<hr/>
Thanks for the tip! Generating the bins worked now. This is the output I got:
<small><pre>
.../wb2pdf/trunk/src$ python3.1 makelinuxbins.py 
[ 1 of 10] Compiling MediaWikiParseTree ( MediaWikiParseTree.hs, MediaWikiParseTree.o )
[ 2 of 10] Compiling Tools            ( Tools.hs, Tools.o )
[ 3 of 10] Compiling MagicStrings     ( MagicStrings.hs, MagicStrings.o )
[ 4 of 10] Compiling MediaWikiParser  ( MediaWikiParser.hs, MediaWikiParser.o )
[ 5 of 10] Compiling TableHelper      ( TableHelper.hs, TableHelper.o )
[ 6 of 10] Compiling WikiHelper       ( WikiHelper.hs, WikiHelper.o )
[ 7 of 10] Compiling MyState          ( MyState.hs, MyState.o )
[ 8 of 10] Compiling WikiLinkHelper   ( WikiLinkHelper.hs, WikiLinkHelper.o )
[ 9 of 10] Compiling LatexRenderer    ( LatexRenderer.hs, LatexRenderer.o )
[10 of 10] Compiling Main             ( pa.hs, pa.o )
Linking pa ...
[4 of 5] Compiling UrlAnalyse       ( UrlAnalyse.hs, UrlAnalyse.o )
[5 of 5] Compiling Main             ( geturl.hs, geturl.o )
Linking geturl ...
</pre></small>
I issued <b>sudo python3.1 installunifont.py</b> next and got:
<small><pre>
Use-my-metrics flag set on at least two components in glyph 1122
Use-my-metrics flag set on at least two components in glyph 1122
Use-my-metrics flag set on at least two components in glyph 1123
Use-my-metrics flag set on at least two components in glyph 1123
Use-my-metrics flag set on at least two components in glyph 1124
[...]
Use-my-metrics flag set on at least two components in glyph 41742
<Font: FreeSerif>
At least one anchor point was lost when pasting from one font to another because no matching anchor class could be found in the new font.
At least one anchor point was lost when pasting from one font to another because no matching anchor class could be found in the new font.
[...]
At least one anchor point was lost when pasting from one font to another because no matching anchor class could be found in the new font.
This is ttf2tfm version 1.5
megafont00:
Glyph  Code   Glyph Name                Width  llx    lly      urx    ury
------------------------------------------------------------------------
    3  00000  .c0x0                      1000      0, -125 --  1000,  875
    4  00001  .c0x1                      1000      0, -125 --  1000,  875
[...]
megafont10fd:
Glyph  Code   Glyph Name                Width  llx    lly      urx    ury
------------------------------------------------------------------------

megafont10fe:
Glyph  Code   Glyph Name                Width  llx    lly      urx    ury
------------------------------------------------------------------------

megafont10ff:
Glyph  Code   Glyph Name                Width  llx    lly      urx    ury
------------------------------------------------------------------------
</pre></small>
In /usr/share/texmf/web2c/texmf.cnf I changed
<small><pre>
% TrueType outline fonts.
TTFONTS = .;$TEXMF/fonts/truetype//;$OSFONTDIR//

% Opentype outline fonts.
</pre></small>
to 
<small><pre>
% TrueType outline fonts.
TTFONTS = .;$TEXMF/fonts/truetype//;$OSFONTDIR//
/usr/share/texmf-texlive/fonts/truetype//

% Opentype outline fonts.
</pre></small>
Next, I created /usr/share/texmf-config/pdftex/config/pdftex.cfg because it didn't exist and added the line <b>map +megafont.map</b>. Also, I looked for pdftex.cfg in /usr/share/ using <b>find /usr/share/ -type f -name "pdftex.cfg"</b> just to be sure it isn't elsewhere, but there was no such file.

I added the line <b>megafont@Unicode@  megafont.ttf Pid = 3 Eid = 1</b> to /usr/share/texmf-texlive/fonts/map/ttf2pk/config/ttfonts.map and issued <b>sudo texhash</b>. Next I started the GUI and just hit "Run" to see if it works, which it doesn't (yet). Here's the output: http://pastebin.com/h4VaZEb1

Especially at the very end, it says
<small><pre>
kpathsea: Running mktexpk --mfmode / --bdpi 600 --mag 1+57/600 --dpi 657 megafont60
mktexpk: Running ttf2pk -q megafont60 657
ttf2pk: ERROR: Missing `='.
megafont@Unicode@  megafont.ttf Pid = 3 Eid = 1
                                                                                     ^
grep: megafont60.log: Datei oder Verzeichnis nicht gefunden
mktexpk: `ttf2pk -q megafont60 657' failed to make megafont60.657pk.
kpathsea: Appending font creation commands to missfont.log.
(see the transcript file for additional information)
!pdfTeX error: pdflatex (file megafont60): Font megafont60 at 657 not found
 ==> Fatal error occurred, no output PDF file produced!
Done
Fehler beim Anzeigen einer URL: Fehler beim Untersuchen der Datei /home/matthias/Masterarbeit/wikitolatex/wb2pdf/trunk/document/main/main.pdf mit fstat(): Datei oder Verzeichnis nicht gefunden
</pre></small>
For readers who cannot read german: "Datei oder Verzeichnis nicht gefunden" means "file or directory not found".

On line 2125 of the log it says:
<small><pre>
LaTeX Font Warning: Font shape `C70/megafont/m/sc' undefined
(Font)              using `C70/megafont/m/n' instead on input line 47.
</pre></small>
So actually, I've got the impression that at least parts of megafont are found, although LaTeX states at the end that it didn't succeed because of megafont60 not being found. Oh, on line 569 there's a similar message, maybe just from the first run.

Despite of this, during the LaTeX processing there are also lots of errors complaining about environments not being closed. This may be related to following warning on line 581:
<small><pre>
LaTeX Warning: Command \! invalid in math mode on input line 63.
</pre></small>
In conclusion, I'm still suspicious about pdftex.cfg not existing beforehand, but I'm not sure if this is really related to every other error. Still, LaTeX complains mainly about megafont60, so of course I should investigate this. Any ideas or comments where I should start?

--[[Spezial:Beiträge/60.12.143.50|60.12.143.50]] 07:42, 20. Jan. 2012 (CET) - Matthias

<hr/>

Hmkay, I replaced
<small><pre>
% TrueType outline fonts.
TTFONTS = .;$TEXMF/fonts/truetype//;$OSFONTDIR//
/usr/share/texmf-texlive/fonts/truetype//

% Opentype outline fonts.
</pre></small>
with:
<small><pre>
% TrueType outline fonts.
TTFONTS = .;$TEXMF/fonts/truetype//;$OSFONTDIR//;/usr/share/texmf-texlive/fonts/truetype//

% Opentype outline fonts.
</pre></small>
Note that I added an additional ";", because it occurs to me that folders are seperated by ";". Afterwards I run "sudo texhash" again and opened the GUI, but the program still complains about megafont60 not found. --[[Spezial:Beiträge/60.12.143.150|60.12.143.150]] 10:35, 20. Jan. 2012 (CET) - Matthias

Installing megafont.ttf is a difficult problem. I essentially learned it by reading some tutorials on the installation of cyberbit.ttf. But possibly you do not need megafont.ttf at all. This is because you seem to be writing in german and thus only need the latin character sets.  In this case you can remove

 \begin{CJK}{UTF8}{megafont}

in 

 wb2pdf/trunk/latex/my-head.tex

and 

 \end{CJK}

in 

 wb2pdf/trunk/latex/my-head.tex

this way megafont.ttf should not be accessed at all. Furthermore if LaTeX is already running the work of the program is basically done. You got a LaTeX file in

 wb2pdf/trunk/document/main/main.tex

and the only problem is to make it compile which you can do manually. Although you will have to become familiar with Latex for that.
 [[Benutzer:Dirk Huenniger|Dirk Huenniger]] 10:52, 20. Jan. 2012 (CET)

My /usr/share/texmf/web2c/texmf.cnf looks like this

 % TrueType outline fonts.
 TTFONTS = .;$TEXMF/fonts/truetype//;$OSFONTDIR//;/usr/share/texmf-texlive/fonts


I think it need to be in one line. But I think you already changed that. Of course you also need to run <code>sudo texhash</code> again.

It is normal that you have to create /usr/share/texmf-config/pdftex/config/pdftex.cfg . It does not exists beforehand.

<hr/>
Okay, the LaTeX file is all I really need. I sometimes use chinese characters, but because of that I switched to XeLaTeX. I'm currently working on my master thesis (laser physics), and this tool will help me very much. My plan is to get those Wikipedia pages that relate to my topic, create one tex file, and annotate everything I don't understand, need to investigate or is otherwise worth being marked. I already did it with one article, and it works (after changing a few LaTeX commands, but maybe I'll automate this process). Thanks a lot for creating wb2pdf! Matthias [[Spezial:Beiträge/60.12.143.181|60.12.143.181]] 17:00, 24. Jan. 2012 (CET)


I am glad that there are people who are able to use my software for their projects, even though the have got a university degree in physics, which might be required to operate my text processing software. It is interesting that you report success with XeLaTeX. I also tied that and did not succeed. So it would be nice if you could write down what you did so it will become more easy for others who also want to use XeLaTeX. In the end I could possible come up with a version of my program in which I you can switch between pdfLaTeX and XeLaTeX by changing a flag in the header. But this is still quite far from now. I would be really nice if you could describe your changes to make it run with XeLaTeX, although I can not guaranty if and when I will integrate it into the main process. [[Benutzer:Dirk Huenniger|Dirk Huenniger]] 17:42, 24. Jan. 2012 (CET)

It might be helpful for you to know that you can combine wikipages to one big wikipage. The wiki engine got a syntax for that 

<pre>
= Laser =
{{:Laser}}
= Stimulierte Emission =
{{:Stimulierte Emission}}
</pre>

See [[w:Benutzer:Dirk Hünniger/Laser]]

<hr/>

My workflow so far was to just take the body of the generated .tex files and fiddle around with the latex code to get them processed. I wrote a sed script that would make some automatic processing like replacing "myitemize" with "itemize" and so on, so that the code, after I fix several remaining issues manually, can be processed with my own header. 

Today I got fed up with it because it just took to long. I started from your hint to comment out the CJK tags in my-head.tex and my-tail.tex. Also, I did following changes:
* comment out some lines in options.tex:
<small><pre>
%\ohead{\headmark}
%\renewcommand{\chaptermark}[1]{\markleft{#1}{}}
%\renewcommand{\sectionmark}[1]{\markright{#1}{}}
%\ofoot[\pagemark]{\pagemark}
</pre></small>
(since the package scrpage2 throws an error, I commented it out and I think as a result the above mentioned commands don't work anymore)
* In packages1.tex, comment out inputenc and textcomp (I just guess that these are not needed with xelatex)
* In packages2.tex, comment out pslatex, fontenc, scrpage2, arabtex, tipa (I just guess that these are not needed with xelatex)
* my-head.tex (blank lines removed to not blow up this page too much):
<small><pre>
\documentclass[11pt,paper=A4,BCOR=12mm,DIV=13,open=any,liststotoc]{scrbook}
\input{../headers/packages1}
\input{../headers/babel}
\input{../headers/svg}
\input{../headers/packages2}
\input{../headers/defaultcolors}
\input{../headers/hyphenation}
\input{../headers/commands}
\input{../headers/title}
\input{../headers/paper}
\input{../headers/options}
\input{../headers/formattings}
%\input{../headers/unicodes}
\input{../headers/templates}
\input{../headers/templates-dirk}
\input{../headers/templates-chemie}
\usepackage{type1ec}      
%\usepackage{CJKutf8}
%\usepackage[overlap, CJK]{ruby}
\usepackage[overlap]{ruby}
%\usepackage{CJKulem}
\usepackage{xltxtra} % added by me
\setmainfont[Mapping=tex-text]{Linux Libertine}
\begin{document}
%\begin{CJK}{UTF8}{megafont}
\raggedbottom
\thispagestyle{empty}
\pagestyle{empty}
%\include{coverfrontpage}
%\cleardoublepage
\pagenumbering{Roman}
\maketitle
\pagestyle{headings}
\setcounter{tocdepth}{\mytocdepth}
\tableofcontents 
%\cleardoublepage
\pagenumbering{arabic}
%\include{kap-vorwort}
</pre></small>
* Replace "pdflatex" with "xelatex" in src/all.py, src/run.py, src/latex.py
* Comment out "f.write("\\usepackage["+ss+"]{babel}\n")" in src/babeldict.py - XeLaTeX doesn't like babel: http://tex.stackexchange.com/questions/27690/problems-with-opentype-options-under-xetex
And that's it. I hope I didn't forget anything. Now Chinese / CJK characters still aren't displayed because when I type them manually, I manually use another font (somehow similar like described here: http://www-scf.usc.edu/~potamite/latch.html or here: http://jalasthana.de/?p=231). If it bugs me too much, I will address this issue later (for now it doesn't bother me).

Thanks for the tip to combine wikipages to one big page! It's time to create an account, I guess. But for today, I'm going to do some more physics instead of computer science (you know, it's actually late evening here).

Also, here's a possible bug report: If I want to process the pages about Taylor's theorem or the heat equation, the script doesn't complete, and top shows me that "pa" is using 100% processor capacity. I'm glad to provide further information if you want to dig deeper into this. [[Spezial:Beiträge/58.100.141.250|58.100.141.250]] 14:32, 2. Feb. 2012 (CET) Matthias

The Problem is known to me. I am currently using a recursive decent parser. In bad cases  it can have a runtime that depends in a factorial (so worse than exponential) way on the lenght of the input file. The ultimate solution is to use a parsing algorithm that guaranties linear time on input length. In trying to do so I found a bug in the Glasgow Haskell Compiler (GHC) which is not fixed but the fix will only become available with the next release of the GHC which will come out in a few month. I also tried to improve the current algorithm. In the current revision the heat equation compiles fine, but the taylor theorem still fail to compile. You may delete you current version and checkout the latest version from svn.

<pre>
svn co https://wb2pdf.svn.sourceforge.net/svnroot/wb2pdf@899 wb2pdf 
cd wb2pdf/trunk/src
python3.1 makelinuxbins.py
</pre>

It is important that you delete the old files before. This will of course also delete you changes, so you should save the somewhere else before deleting them. You can somehow work with some patch and diff mechanisms to get your local changes applied again to the current revision, but I don't know enough about that myself. What you essentially got right now is revision 869 with your local changes. You can also download revision 869 original version. You can create patch with some programms bases on diff. You can then apply that patch to version 899, which will hopefully somehow work.[[Benutzer:Dirk Huenniger|Dirk Huenniger]] 15:05, 2. Feb. 2012 (CET)

I modified the article on taylors theorem in the english wikipedia, so it compiles correctly now. Fell free to tell me if you face more problems.[[Benutzer:Dirk Huenniger|Dirk Huenniger]] 16:05, 2. Feb. 2012 (CET)

The Problem about taylor is essentially that you have to write

<pre>
<nowiki>
''e''<sup>''x''</sup>
</nowiki>
</pre>

But you must not write

<pre>
<nowiki>
''e<sup>x''</sup>
</nowiki>
</pre>

The rule is that environments have to be closed in the reversed order in which they were opened. So it is A-Open B-Open B-Close A-Close and never A-Open B-Open A-Close B-Close. Its exactly the same concept as brackets in mathematical formulas. My Program tries to correct for improper bracketing but it is not good at it. [[Benutzer:Dirk Huenniger|Dirk Huenniger]] 19:04, 2. Feb. 2012 (CET)

I worked on the problem and as of revision 927 do get an output on the article on taylors theorem with the original (wrong) bracketing after several minutes. Still I seemed to have cause a lot of collateral damage in creating this revision, so the output does not look desirable.  [[Benutzer:Dirk Huenniger|Dirk Huenniger]] 12:52, 5. Feb. 2012 (CET)

As of revision 935 we produce good looking output on the original version of the english wikipedias article on taylors theorem. Especially improper bracketing is taking into account in a correct way. [[Benutzer:Dirk Huenniger|Dirk Huenniger]] 13:02, 6. Feb. 2012 (CET)

I am going to release revion 937 as version 4.2 fixing this issue.

I released version 4.2 as revision 950. So this problem is solved now.[[Benutzer:Dirk Huenniger|Dirk Huenniger]] 09:13, 8. Feb. 2012 (CET)

Your bug report about requiring to parse improperly bracketed expression gives rise to an interesting theoretical result. Allowing arbitrary improper bracketing implies via the pumping lemma that the grammar is not context-free. This rules out a large class of parsers and parser generator. Currently I am using a handwritten parser that allows this case, but you essentially forget about any parser generator. [[Benutzer:Dirk Huenniger|Dirk Huenniger]] 13:34, 11. Feb. 2012 (CET)