Brackets [] wrongly displayed in Intellij when focused (bold)
hrach opened this issue · 78 comments
When I put cursor to brackets, they get bold and the bottom line is not visible at all.
Windows, Hack-v2_015-ttf, size 12, changing line-spacing does not help.
FWIW i do not see this issue, lntellij 14.1.5 - hack v15
Any idea how to debug it? PhpStorm, PS 141.x
check #45 maybe, i had some caching issues when i upgraded the font from the previous version that had this behaviour.
Try issue report #100 where a similar problem was identified. They reported an issue on the JetBrains issue tracker. It sounds like this was fixed as of the v2.015 release.
- I have fresh new laptop, so, from the beginning I am using hack v15.
- though, I tried to uninstall the font, invalidated caches in PS, nothing helped
- I tried to upgrade to PS 9.5 EAP, didn't help.
Important note: previously mentioned bugs seem to be caused by current line highlighting, which overlay-ed the bottom parts of letters on the previous line. My issue seems to be buggy in every case when the bracket is bolded.
But as mentioned in the other bugs, changing font-size to bigger don't help, however, 11 works ok.
PhpStorm 10. v18, bug is still present.
Thank you
just reporting in that this is still present in 2.020 release, i have created a new ticket on the intellj side https://youtrack.jetbrains.com/issue/IDEA-155229
@jorgheymans mind trying the 25% line spacing script (see top of readme) to see if it addresses this in IntelliJ?
i'm on windows only here sorry , @hrach maybe ?
happy to build them for you if that is the problem.
sure i can try out whatever zip you make available no problem.
Give these a try. This slightly increases the default line spacing across all variants.
Do you know how to clear the Windows font cache and confirm that there are not duplicated font files with font upgrades?
well i remove them from the font dir first, then drop the new fonts in the windows font dir is that sufficient? also restarting intellij ofcourse.
In any case it did not solve the issue it seems.
I did the same as @jorgheymans with the same result. (Windows 10)
Not IntelliJ IDEA specific, NetBeans has the same problem with bold Hack font:
Also note incorrect glyphs for bold italic: https://bugs.openjdk.java.net/browse/JDK-8078382.
Also note incorrect glyphs for bold italic
@CrazyCoder This is sounding like a caching / font duplication issue. We saw this when Windows users' systems did not clear the previously installed versions (this is the default unfortunately). Are you willing to try our new Windows installer to see if it addresses this?
cc: @texhex
@CrazyCoder It looks like there may be clipping of the parentheses glyphs in those images as well. Is that the case?
Hmm, this was the first time I installed Hack font on this system, however I tried the installer and it has fixed the problem. Weird.
@CrazyCoder Windows makes our head ache :) The current Windows installer contains v2.019. v2.020 should be available sometime in the next few days. I would recommend the 2.020 update to Win users. We addressed the hints in some important glyphs (including the zero) that should make a difference on the platform.
@hrach @jorgheymans want to give the installer a try? it looks like it fixed the bracket problem for @CrazyCoder
@CrazyCoder can you let me know how you performed the initial install?
@hrach @jorgheymans @CrazyCoder
The Hack Windows installer has been updated to install Hack v2.020. Please give it a try.
CC: @chrissimpkins
is there a simple way to script this for testers on Windows?
Sorry, but I don't understand what should be scripted? If there is this special character issue?
can you let me know how you performed the initial install?
Right click on .TTF files, Install.
@texhex Wondering if there is a way to provide the functionality of the install tool without the need to compile the installer for those who help test specific issues in the fonts. This wouldn't be used for distribution, but to provide assurance that the problems that we identified (and are addressed with the installer) are not responsible for the issue persistence when testers attempt fixes through new installs.
Right click on .TTF files, Install.
Thank you! Does that open an install wizard or is that all of the communication from the system that you receive?
Does that open an install wizard or is that all of the communication from the system that you receive?
It showed some progress and that was all.
Wondering if there is a way to provide the functionality of the install tool without the need to compile the installer for those who help test specific issues in the fonts.
I really don't know if this would be of any help to duplicate the functionality in a different tools by the same coder. If I made an error in the installer by misunderstanding the inner works of Windows, I will most likely make the same mistake again. Just in a different tool/language.
What I could come up is to write a test for Xteq5 (Disclaimer: I'm the creator) that performs a test if the registry information for all fonts match the font files on disc. Also, if all fonts on disc are registered (to the registry).
But I don't know if how many percent of the issues we have seen would be detected by this.
It showed some progress and that was all.
Thanks for the explanation.
This is rather strange because so far our understanding was that the updates can lead to problems, but we had the impression that the very first installation should always work. I do not have an explanation why this has failed on your system, expect that somehow the TTF files were damaged.
We have seen this also, but only that the TTF files were damaged later on. Not directly after copying them from the ZIP.
@texhex Sorry, I think that I am not being clear (or I misunderstand your response). What I would be interested in is a straightforward reusable tool (ideally a script) that performs the font maintenance (cache clear/duplicate font cleanup/etc) that has been problematic on Windows and could be used by those who report new Windows issues prior to attempts to install new font builds that are intended to correct the problem. The bug fixes would take place in the fonts, not the installer/install tool. This would allow us to point Windows users to this tool, run it to clean their current font state, then install the test fonts in order to verify whether they addressed a problem. We could do this with the compiled installer but it seems unreasonable to compile a new installer each time we create a set of testing fonts. Alternatively, we could pull the information that you provided out of the issue thread and ask Windows users to complete those tasks prior to testing a new set of fonts.
As a soapbox sidebar, I find it astonishing that there is not such a tool around. The commercial foundries must have proprietary tools for this. I do not believe that this has not been appreciated before now...
or they aren't testing on Windows ;)
@CrazyCoder Because we are still clueless what was going on, would you mind uploading the setup log file to the repository of the Hack Windows installer repository?
If you agree, please do the following:
- Start -> Run -> %temp%
- Locate the file called
Setup Log 2016-04-26 #xxx.txtwhere XXX should be 001 - Double-click it and look into it to make sure it's the setup log from our installer. In the third line it should say something about
HackWindowsInstaller.exe - Check the file for any information that you do not want others to see. For example, line 3,4 and 10 contain the paths where the setup is started from and how your TEMP folder is called. Just delete these lines. The other lines should be safe and only contain common information.
- Follow this link to open a new issue in the Hack-Windows-Installer repository
- Paste the entire text (minus the lines I noted above) there and we can check if we can get any information what was going wrong.
Thanks!
FYI just ran the installer, no difference bug is still present.
@jorgheymans Sorry to hear that. Can you maybe post the setup log file (as I have described above) to the HackWindowsInstaller repository? Also, do you have access to any other non-JDK based tool where you create the problematic glyph(s) with Hack Bold and see if the display error also displays there?
i tried locating the installer file but it was not there, i guess %temp% is cleared out by the restart that the installer requires ?
Using the font in gvim as well, it does not have the same issue
The setup log should survive a restart (at least it did on all my test machines). Strange. I will think about how to duplicate the important information from the setup log to the installation folder later on.
Given that there are no issues in GVIM, could it have something to do with the Text Antialiasing settings in IntelliJ?
This bug report comment shows a picture of the settings of IntelliJ. Do the comments there match your issues? Means, if you change either from Hack Bold to Hack Regular and/or change the Antialiasing settings does it make any difference?
ok so using the testcase (modified for Hack see below) linked in the bug report i can reproduce it in plain Swing, so it has nothing to do with intellij but it's a windows bug.
Also, from that bug report, if i add -Dsun.java2d.font.scaler=t2k to my intellij startup options the problem goes away ! So it was a windows bug all along it seems.
import javax.swing.*;
import java.awt.*;
import java.util.Arrays;
public class EqualsGlyph {
private static final String FONT_TO_TEST = "Hack";
public static void main(String[] args) {
SwingUtilities.invokeLater(new Runnable() {
@Override
public void run() {
final JFrame frame = new JFrame("Equals Glyph");
frame.setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
frame.add(new MyComponent());
frame.setSize(200, 100);
frame.setVisible(true);
frame.setLocationRelativeTo(null);
SwingUtilities.invokeLater(new Runnable() {
@Override
public void run() {
String[] fontFamilies = GraphicsEnvironment.getLocalGraphicsEnvironment()
.getAvailableFontFamilyNames();
if (!Arrays.asList(fontFamilies).contains(FONT_TO_TEST)) {
JOptionPane.showMessageDialog(frame,
FONT_TO_TEST + " font is not installed.",
frame.getTitle(), JOptionPane.ERROR_MESSAGE);
}
}
});
}
});
}
private static class MyComponent extends JComponent {
@Override
protected void paintComponent(Graphics g) {
Graphics2D g2d = (Graphics2D) g;
g2d.setRenderingHint(RenderingHints.KEY_TEXT_ANTIALIASING,
RenderingHints.VALUE_TEXT_ANTIALIAS_LCD_HRGB);
g.setFont(new Font(FONT_TO_TEST, Font.BOLD, 12));
FontMetrics fm = g.getFontMetrics();
g.drawString("[]", 5, 5 + fm.getAscent());
}
}
}
👍
If this is in fact the fix, JetBrains would be interested in hearing this. We have filed multiple issue reports on the tracker for these rendering problems and to my knowledge they have not been resolved.
@chrissimpkins: I think they already know and have proposed a fix: http://mail.openjdk.java.net/pipermail/2d-dev/2016-April/006674.html
Full route: IDEA-155229: highlighted array brackets only partially shown links JDK-8146035 Windows - With LCD antialiasing, some glyphs are not rendered correctly which links to OpenJDK 2D-Dev PATCH JDK-8146035: Windows - With LCD antialiasing, some glyphs are not rendered .
thanks to both of you for digging around on this issue. This is very good to know. Will leave this open until they resolve the problem.
Since this most likely applies to almost all jetbrains products a mentioning in the README is in order i think. You could just copy what the intellij guys concluded in the issue:
Can we safely say that this is Windows only?
Yes, I would say so. But I still try to wrap my head around this.
If I understand this issue correctly, Windows expects several calls in a particular order or they will behave "strange". This does not happen on any other system. So, a Windows bug.
But what I do not understand that it seems to happen only when using "Hack Bold" and when using anti aliased text and highlight it (see the first attached GIF at IDEA-155229). But there are two things that I can't understand:
- One user reported it helped to use the installer This shouldn't be the case when the API has a bug.
- The proposed fix on the OpenJDK 2D Mailing list says I couldn't reproduce the issue for fonts bundled with Windows. This does not make sense to me at all. An API bug should also be reproducible with any font.
all good points but honestly there is so much complexity and layers involved that for such cases i have learned to not wanting to understand it anymore. Until i started tracking this project i always thought a font was something anyone with a bit of drawing skills could cook up in a couple of hours, go figure :-)
Do you happen to know what the font.scaler=t2k does? It could be something in our OpenType tables that the Java renderer does not like with the default settings.
@jorgheymans Yes, bugs of this sort are a PITA. But even if you give up on this issue, maybe we don't :-).
Would you mind to extend your example to include both Hack Bold and Hack Bold Italic as well as how it looks like when you use a different VALUE_TEXT_ANTIALIAS value as well when turning AA completly off. A screenshot would be a plus ;).
@texhex see attached
i whipped up a small java class that accepts a string as argument and displays it using all aliasing text hints in the four font variations :
(omitting the parameter will use []={}_() as string argument)
import javax.swing.*;
import java.awt.*;
import java.util.Arrays;
public class EqualsGlyph {
private static final String FONT_TO_TEST = "Hack";
private static String DEFAULT_TEST_STRING = "[]={}_()";
public static void main(String[] args) {
SwingUtilities.invokeLater(() -> {
if (args.length == 1) {
DEFAULT_TEST_STRING = args[0];
}
final JFrame frame = new JFrame("Equals Glyph");
frame.setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
frame.add(new MyComponent());
frame.setSize(1000, 1200);
frame.setVisible(true);
frame.setLocationRelativeTo(null);
SwingUtilities.invokeLater(() -> {
String[] fontFamilies = GraphicsEnvironment.getLocalGraphicsEnvironment()
.getAvailableFontFamilyNames();
if (!Arrays.asList(fontFamilies).contains(FONT_TO_TEST)) {
JOptionPane.showMessageDialog(frame,
FONT_TO_TEST + " font is not installed.",
frame.getTitle(), JOptionPane.ERROR_MESSAGE);
}
});
});
}
private static class MyComponent extends JComponent {
private Object[] ANTI_ALIASING_HINTS = new Object[]{RenderingHints.VALUE_TEXT_ANTIALIAS_DEFAULT,
RenderingHints.VALUE_TEXT_ANTIALIAS_GASP, RenderingHints.VALUE_TEXT_ANTIALIAS_LCD_HBGR,
RenderingHints.VALUE_TEXT_ANTIALIAS_LCD_HRGB, RenderingHints.VALUE_TEXT_ANTIALIAS_LCD_VBGR,
RenderingHints.VALUE_TEXT_ANTIALIAS_LCD_VRGB, RenderingHints.VALUE_TEXT_ANTIALIAS_OFF,
RenderingHints.VALUE_TEXT_ANTIALIAS_ON};
@Override
protected void paintComponent(Graphics g) {
Graphics2D g2d = (Graphics2D) g;
FontMetrics fm = g.getFontMetrics();
for (int i = 0; i < ANTI_ALIASING_HINTS.length; i++) {
g2d.setRenderingHint(RenderingHints.KEY_TEXT_ANTIALIASING, ANTI_ALIASING_HINTS[i]);
g.setFont(new Font(FONT_TO_TEST, Font.PLAIN, 12));
g.drawString("HINT: " + ANTI_ALIASING_HINTS[i].toString(), 5, 15 * i + 5 + fm.getAscent() + i * 120);
g.drawString("NORMAL " + DEFAULT_TEST_STRING, 5, 15 * i + 20 + fm.getAscent() + i * 120);
g.setFont(new Font(FONT_TO_TEST, Font.BOLD, 12));
g.drawString("BOLD " + DEFAULT_TEST_STRING, 5, 15 * i + 40 + fm.getAscent() + i * 120);
g.setFont(new Font(FONT_TO_TEST, Font.ITALIC, 12));
g.drawString("ITALIC " + DEFAULT_TEST_STRING, 5, 15 * i + 60 + fm.getAscent() + i * 120);
g.setFont(new Font(FONT_TO_TEST, Font.ITALIC + Font.BOLD, 12));
g.drawString("BOLD ITALIC " + DEFAULT_TEST_STRING, 5, 15 * i + 80 + fm.getAscent() + i * 120);
}
}
}
}
but that assumes you know how to compile and run java. If that is not the case download the compiled version :
hackui.zip
assuming you have java8 installed you can run it with
"C:\Program Files\Java\jdk1.8.0_91\bin\java" -jar hackui.zip
or
"C:\Program Files\Java\jdk1.8.0_91\bin\java" -jar hackui.zip "MY TEST STRING"
or using the workaround:
"C:\Program Files\Java\jdk1.8.0_91\bin\java" -Dsun.java2d.font.scaler=t2k -jar hackui.zip "MY TEST STRING [][][]"
I hope this will help you investigating further, or you could use it to detect future regressions maybe
Wow, thanks! This is very useful and much appreciated. And it will be very useful to easily detect rending issues, just as you proposed.
Under which license do you plan to release this source? Would MIT be OK for you? I'm asking because I think it would make sense to push this code to a repository so it is not "lost" in this issue comment.
@chrissimpkins Please have a look at the picture, does it help to identify any hinting issues? Also, please let me know if you agree with moving the code to a separate repository.
@texhex it's just an adapted version of the test class that was used in the openjdk ticket, you can use it or modify it as you want. I would be happy to improve it still if needed. Good point about moving it outside of the ticket somewhere, probably could be made more visible.
Now, what I noticed just now looking at the picture is that the brackets are only cut off in the LCD HRGB and LCD HGBR aliasing mode, other modes just work fine. Something to do with subpixel rendering perhaps. The rendering hints are described here https://docs.oracle.com/javase/7/docs/api/java/awt/RenderingHints.html in case it helps..
Yes, LCD HRGB and LCD HGBR screw up the lower part of [] when using Bold, while DEFAULT, LCD VBGR and LCD VRGB seems to mess up [ when using Normal (Regular).
VALUE_TEXT_ANTIALIAS_GASP and "Antialiased Text Mode" seem to be fine always.
Could you please explain what is this T2K scaler is about?
When I understood it correctly (Source#1 Source#2), the JDK now uses the Freetype font scaler which seems to have problems with some fonts on Windows when using native scaling, while the T2K scaler is the older font scaler using by the Sun JDK which doesn't had this issue?
I would be happy to improve it still if needed.
Awesome! Let's see what the godfather of Hack says to all of this.
I'm gonna make them an offer they can't refuse...
I'm gonna make them an offer they can't refuse...
I have the slight feeling I will regret calling you godfather of Hack...
I have the slight feeling I will regret calling you godfather of Hack...
Ha!
Re: How to address this in documentation
@texhex @jorgheymans I read the Java issue report last evening and it appears that they closed this as fixed in the patched JDK if I understood it correctly. Do you happen to know if it is possible to define a different JDK in the JetBrains editors? If so, perhaps we provide users with this workaround in the docs.
Would either of you be interested in submitting a PR to update our README with the information that Win + JetBrains users need based upon your thoughts?
Re: the new Java tool
@jorgheymans Jorg, would you be willing to push this tool (or some variant of it that @texhex and you feel will best help to address our Java woes) to a repository and license it so that we can either point users to the tool in order to troubleshoot similar problems or perform this testing ourselves?
Further Info
Perhaps we could dig in to the Java documentation on these glyph rendering classes too. The drawString() method on the Graphics2D class and the getAscent() method on the FontMetrics class may yield something that I can work with. I would be very interested to know what vertical metric the getAscent() method is retrieving from the font because there are multiple OpenType table values (that can differ within the same font and differ in approach between fonts) that are options for use. If this is the property that is being used to determine line height and between line vertical spacing we might be on to a solution via the Hack metrics. It remains to be seen whether we can modify this value to address this situation, but at least we know.
@jorgheymans and thank you for this fantastic Java tool by the way! Much appreciated! I am learning about Java font rendering from your code.
Addendum to #129 (comment):
Is it possible, and would it be worthwhile, to add a header in this reporting tool that displays what default antialiasing settings the user's system is using or is this uniform across Windows releases?
@texhex your understanding about the T2K scaler seems correct.
@chrissimpkins you can choose whatever license you want for that class. Should it not stay within the hack repository somewhere ? Otherwise just let me know where to push it to.
About the default antialiasing settings: if there are no rendering hints specified for the text antialiasing then it falls back to the setting specified for all antialiasing, the setting for this is implementation dependent . It would take a decent amount of digging in the jdk sources to figure out what these settings are, i do not think they are meant to be exposed on the jdk level.
Windows and the jdk are two different things, there is no such thing as default settings on the jdk level because every java program is expected to set these settings as they see fit, at least that is my understanding.
In terms of switching jdk, this is doable for the masses as long as its a released version. The jdk used for starting intellij does not have to be the one used for compiling the project . We could add all this in the README. I will have a look to send a PR the next few days to explain all this in the README somewhere.
every java program is expected to set these settings as they see fit
I see. Thanks
In terms of switching jdk, this is doable for the masses as long as its a released version
This is a potential solution then, albeit a very onerous one to address your typeface display...
I will have a look to send a PR the next few days to explain all this in the README somewhere.
That would be great if you are willing to do it. If not, please simply let me know what you feel users need to know to work around this issue. Our README is very long, so the more concise the better, particularly since this only applies to a subset of users.
Should it not stay within the hack repository somewhere ?
I would prefer to expose this in its own repository so that we avoid cluttering the issue report threads here with tool issues. I would be happy to host it in our Source Foundry organization which is where we keep our other project tools, or feel free to push to your own personal account and we would be happy to send people your way when this tool is needed.
Also, can you confirm that this block:
if (args.length == 1) {
DEFAULT_TEST_STRING = args[0];
}modifies the default constant string defined at the top of the file with the first argument to the executable? This would definitely be a desired feature so that we can test other glyphs that prove problematic.
We could even consider making this available as a general tool for typeface developers and provide a way for them to specify the typeface to be tested on the CL. Some thoughts. We can start here for now.
Let me know how you would like to approach the repository. Thanks again for all of your efforts. Will acknowledge these contributions on our contributors list.
ok for me to host that class somewhere in the source foundry repo.
Indeed, the test class allows for a string to be passed on the command line :
"C:\Program Files\Java\jdk1.8.0_91\bin\java" -jar hackui.zip "[#] __ [#]"
will render the canvas using "[#] __ [#]" (without the quotes).
i was thinking as an improvement it would also read text from a file, for those characters that are difficult to pass on the command line. Also capturing the canvas automatically into a jpg would allow for easy regression testing.
@jorgheymans sure sounds great. what would you like to name it?
Like the file read idea. All sounds good!
how about "java glyph tester" -> "a tool for testing the JDK font rendering behaviour"
@jorgheymans great! here you go:
capturing the canvas to a jpg...
Through this tool or an external tool? Is there a simple way to do this in Java?
sure it's supposed to be only a couple of lines going through ImageIO , i will have a look at it.
Sounds good. This will be a helpful tool. Thanks Jorg!
@jorgheymans you should have write access on the Java Glyph Tester repository now. Let me know if you have any issues with it or if there is anything that I can do to help.
cc: @texhex
@jorgheymans Thanks for all the time and effort you put into this Jorg, I'm very sure this tool will help us a lot to get the Freescaler under control or at least prove that it's a Windows/Java issue and not an issue of the font.
Access is working fine, just did initial commit.
@texhex no worries 👍
@jorgheymans great! let us know what, if anything, we can do to help.
To try to finalize this issue, are we in agreement about a workaround here?
yes, i will send a PR this week sometime to the README explaining what is going on and how to workaround. (but it seems there is a PR pending not sure i have to wait for this one to be applied first). How about i put it in a README-intellij.md , catches the attention of those using the tool and others can just ignore it ?
No rush on this. We can wait until the revisions on the README are complete. Just wondering what the approach will be.
It looks as if this issue is fixed in latest intellij 2017.1 with jdk8_121, hurray !
Closing as fixed based upon comment in #129 (comment)