support new GraphicsFormat JRXML (JaperReports)

[elvisbegovic]

Thank to @manuelbl , SwissQRBill library is small, well designed and easy to use.

Here is a feature request :

Description

When users have to "store" big amount of PDFs (1000-100000 pdf) : images and fonts are the weaks points of a PDF.

Today we have three option with SwissQRBil:

1. If we output PNG (300 dpi) the image is > 100-150 Ko. (ex 150 * 10'000 pdf = 1500 mb)
2. If we output SVG it is 10-15 Ko but converting it to a 2d context on PDF (it will increase PDF about 100-150 Ko for each SVG) (12 * 10'00 pdfs = 120 mb)
3. If we output PDF it is about 4 Ko which is awesome and insane, and it is (I think) the best choice if we deal / create invoice with PDF. But sometimes we cannot concat/add pdf if we use a tool creating reports like JasperReports. (4 * 10'000 pdfs = 40 mb)

Proposed solution

Add .JRXML export (as subreport) which is part of JasperReport opensource format. We can add subreport as Datasource in a jasper template and jaspertemplate filling will choose where to put subreport.

If we want keep this lib without other dependency we can create another maven dep with all other type of export like :

JasperReports is well known and wide used and this will make SwissQRBill the only one supported in JasperReports.

What you think ?

[manuelbl]

I‘m not familiar with JasperReports. So can you provide a more detailed description of how it would look like?

• Would the QR bill be generated as a completely independent file and then included into the main report?
• Would the QR bill code be run independently or be somehow integrated into JasperReports?
• What graphic operations does JasperReports offer to draw QR codes, dashed lines and scissors?
• Do you have a useful sample of how a QR bill jrxml file would look like?

I find it surprising that you experience such a file size growth when converting from SVG to PDF. This must either be due to adding fonts or converting the SVG image to pixel graphics. In any case, it is specific to the tools you‘re using and not inherent to the task.

[elvisbegovic]

1. Yes file will be completly different file, not depend to a main report. This is called "subreport", like actually "png" or "svg"
2. It is possible to design subreport with (TIBCO Jaspersoft Studio) that output .jrxml (then convert it once to .jasper which is Java bytecode) like that it will generate subreport faster.
3. and 4. Afaik Jasper Studio (if not created fully by java code) offer easyway to design the report in xml format, here you are to answer to your question here is an example :
   

Regarding SVG, I mainly think because SVG is converted to 2dcontext (convert img to pixel graphics). It is not due to font because 7 svg will increase 7x100ko+ for a pdf of 7 pages. Pdf not deal well with not-pixel things like SVG but actually it is the only way I found to works with jasperreports, generate with SwissQRBill a svg then sent it thought code to the report in a IMAGE:

[manuelbl]

According to my analysis, the large size of the resulting PDF file when including a SVG image into a Jasper report is not due to the conversion to pixel graphics image. There is no pixel graphics image in the resulting PDF. Instead, it is due to the fact that text is converted to outlines. So each letter is drawn using graphics operations such a line, bezier curve and fill. If a letter appears twice or more, then these graphics operations will be repeated many times. I can't tell if the SVG to Graphics2D, or the Graphics2D to PDF converter causes it.

I've also looked into how to create a subreport. The QR bill uses a library that generates basic graphics operations such as drawing text, creating a path from lines and cubic curves, and filling or stroking it. Unfortunately, JasperReports has no matching facilities. It can do text and straight lines. That's it.

So the best approach would probably be to direct text operations to a .jrxml file and all other operations into a SVG file, which is then included into the .jrxml file. So each QR bill would result in 2 files: a .jrxml and an SVG file. I think that's doable but ugly.

Factoring out the QR code can be skipped as a QR code generated natively from JasperReports results in a bigger file than a QR code generated with the QR bill library and then included as a SVG. (Some of the size optimizations done in the QR bill library are lost, but some survive.)

I've also looked into creating a JasperReports extension so that QR bills can be generated without the need for a subreport. But I couldn't figure out in reasonable time how this would be done and if it is possible at all. And if it turns out that the Graphics2D to PDF converter causes the problematic font conversion, it wouldn't even result in smaller PDF files.

Overall I'm not sufficiently familiar with JasperReports. Even if I create something that creates a .jrxml file, I couldn't judge if it is actually working and useful.

The QR bill library is extensible to a certain degree and there is no need to integrate the .jrxml generation directly into the library. Thus, I propose you create it yourself by implementing a new Canvas class. You don't need to modify the library. Instead:

• Copy the SVGCanvas class and call it something like JapserCanvas.
• Remove ByteArrayResult, toByteArray(), writeTo() and saveAs()
• Modify the constructor according to your needs. It should open a .jrxml and a .svg file (reusing stream) and write the header to both files.
• Modify close() such that it writes the closing tags to both files and closes them
• Modify putText() such that it writes JasperReports XML tags to the .jrxml file instead of the .svg file

The only challenge will be the transformation matrix. Since all the text is drawing without rotation and scaling, it's probably sufficient to remember translateX and translateY from the last call to setTransformation(). And the coordinate system is likely different: the QR bill library uses a PDF like system with the Y-axis point upwards, while most other graphics system have a Y-axis pointing downwards. So y must be negated (also see first line of putText()).

Finally, generate the files like this:

try (var canvas = new JasperCanvas("qrbill.jrxml", "qrbill.svg")) {
    QRBill.draw(bill, canvas);
}

[manuelbl]

After thinking some more about it, there might be a different and simpler approach for JasperReports:

The QR bill payment slip could be generated with a fixed Jasper report/subreport (instead of an individual one for each invoice). The software seems capable enough. The QR bill library would be used to validate the billing information, generate the QR code text and format all the text field. It would focus on the most common use case in such a scenario, which is a bill with a known debtor and a known amount.

In particular, the the library would:

• generate the text for the QR code
• format the creditor account and creditor address as a single multi-line text
• format the amount
• format the debtor address
• format the reference
• format the additional info

I've experimented with JasperReports somewhat and get good results except for some funny behavior that a JasperReports specialist could likely fix.

What do you think about this approach?

[elvisbegovic]

Dear @manuelbl

thank you a lot for smart inputs.

let me try to answer.

when you say

So each letter is drawn using graphics operations such a line, bezier curve and fill. If a letter appears twice or more, then these graphics operations will be repeated many times

Yes You are right, theses are called « x objects form » and represent 95% of the PDF due to duplicates same graphics object instead reference them. Not related to this issue request but I created issue on jasper project https://tinyurl.com/Compress-x-object-form I think this could be processed/cleaned on some exportPdf() method of the lib or the lib they use.

About JasperCanvas, to be honest I don’t find JasperCanvas() solution DX friendly. Too many internal concept.

about your last comment : this sound better, but what you mean by :

The QR bill payment slip could be generated with a fixed Jasper report/subreport (instead of an individual one for each invoice).

Do you mean we should design our subreport.jxml using JasperStudio Soft, create fixed report and java use SwissQRBill to generate all data + send info in parameters ? And the lib validate+expose all required data in a java pojo? If so them this seems be the best solution if we can’t expect from lib to generste fully this jxml.

When you say, you experienced. Do you mean you create jasper report .xml with JasperStudio soft?

[manuelbl]

I've indeed worked with Jaspersoft Studio and generated a report that displays a QR bill for the each line in a CSV file. It displays the text in the correct positions, adapts to longer, shorter or missing text, displays the QR code, the Swiss cross, and the dashed lines.

I haven't managed to create a working sub report. And even in the current report, the static texts are omitted on some pages. The software is a mystery to me.

Do you have the experience to fix the problems in the report, and possibly even create a setup with a working report?

[elvisbegovic]

@manuelbl Yes of course I can manage to create a .jxml subreport but iso20022 specs are out of scope to me and I don’t know what number/position of different text zone (textfields) we need design (using as you did jasper soft studio).

Can you share what you did in jasper soft to understand what is the problem you are facing

if by « create working report » you mean create a hello-textfield in report.jxml that contain another subreport.jxml with hello2-textfield, let me know

[manuelbl]

@elvisbegovic I'm happy to share the reports. It'll take me another day to create a test dataset I can release with it.

[manuelbl]

The attached zip file is the partially working report. It consists of:

• qrbill.jrxml: JasperReports source file
• SampleData.csv: A CSV file to be used as data source (I've checked Skip the first line and selected UTF-8 as encoding when setting up the data source.)
• swisscross.svg: A SVG file for creating the Swiss cross in the center of the QR code.
• RawData.csv: For illustration purposes only (see below)

report.zip

The report generates a QR bill at the bottom of each page. However, there is a problem: starting on page 2, the static texts (labels) in the right column are no longer printed and the vertical movement to close gaps if parts of the bill are omitted does not happen. In the left column, it all works fine.

I was also unable to convert it into a subreport.

The overall setup would be that the QR bill library is fed with the raw data as shown in RawData.csv (in the form of Bill instance) and then generates the preformatted data for JasperReports (as shown in SampleData.csv).

The current limitations are:

• Labels are all in German
• Debtor must be provided (no support for empty field with corners)
• Amount must be provided (no support for empty amount with corners)
• Alternative schemas are not supported
• Use a smaller font if the text does not fit.

This limitations could all be removed if there is need.

If you can fix the problem in the right column, it would be very helpful. And a conversion to a subreport would probably also be useful.

[elvisbegovic]

@manuelbl please apologize the delay.
I tried your jasper indeed some feature is missing.

but I tried your jasper on all pages and output pdf is bigger because it is a PNG.

By the way our current implementation putting SVG from your lib, internal jasper 3th library (itext) using SVG to context2d because and create many X FormObject. The only problem is that this is not optimized, when I use pdf-Pro tout optimize PDF it remove all duplicates xformobject and the pdf is really small.

i will continue to search how to fix the size issue
If we can ask to jasper lib to export jasper and remove duplicates it will be the best, I come back to you.

the issue here is PNG for each QR. The best approach will be continue to keep QRcode only as SVG element all others as textfield like your JasperReport.

by the way: do you alread tried to export more humanreadable all infos composing qrbill , make methods public and let user access all methods creating QR.

[manuelbl]

but I tried your jasper on all pages and output pdf is bigger because it is a PNG.

How exactly do you generate the PDF? And how big a PDF file for a single QR bill? I export the PDF generated with Jasper report attached above directly from Jaspersoft Studio. The result is a PDF file of about 20KB. And it doesn't involve any PNG. Within the PDF, most of the size is taken up by a content stream drawing the QR code.

For a Jasper report with the entire QR bill embedded as an SVG, my analysis also shows a completely different result than what you are saying. When exported from Jaspersoft Studio, the resulting PDF contains no PNG or image. Instead, it contains a large stream drawing the outline of each character individually and then filling it (instead of using text operations).

Can you describe how exactly you are generating the PDFs because we get result that differ considerably. In particular, I don't get anything related to PNG or XFormObjects of relevant size.

by the way: do you alread tried to export more humanreadable all infos composing qrbill , make methods public and let user access all methods creating QR.

I was hoping that you can have a look at provided Jasper report and fix the issue with the missing labels. If it can't be fixed, then I'm not sure it is worthwhile adding to the QR bill library.

[manuelbl]

Further investigation has confirmed that the main contributor for the huge file size is that text in SVG is converted into shape outlines. If the entire QR bill payment slip is added as an SVG, the text will contribute about 70KB to the PDF (per payment slip).

The reason for this is the Batik library. Jasper Reports uses it to parse the SVGs and to render them to a Graphics2D instance of OpenPDF. In this setup, Batik will always convert all text glyphs to shapes.

If both the text and QR code are generated with Jasper Reports built-in features, then the text is small but the QR code contributes about 20KB to each payment slip.

So I've created a test with a custom QR bill renderer and published it as an example: https://github.com/manuelbl/SwissQRBill/tree/master/examples/jasper_reports_rendering. It implements a new class that can be used in a data source and passed as a field to a Jasper Report image. It will render the entire QR bill payment slip, with all options.

As it no longer requires the Batik library and as some of the QR code optimizations of the QR bill library are preserved, the resulting PDF size is at about 12KB per payment slip. That is quite reasonable.