Illegal State issue from initdb
jweaver-pfi opened this issue · 44 comments
Hi, I'm attempting to use otj-pg-embedded-0.11.0 (as downloaded from the maven repository) on a windows 10 system and I am getting the following when I attempt a simple junit test:
java.lang.IllegalStateException: Process [C:\Users\john\AppData\Local\Temp\embedded-pg\PG-20610842f7206a9abbecdcf12709c0f8\bin\initdb.exe, -A, trust, -U, postgres, -D, C:\Users\john\AppData\Local\Temp\epg3467157523703904606, -E, UTF-8] failed
at com.opentable.db.postgres.embedded.EmbeddedPostgres.system(EmbeddedPostgres.java:558)
at com.opentable.db.postgres.embedded.EmbeddedPostgres.initdb(EmbeddedPostgres.java:223)
at com.opentable.db.postgres.embedded.EmbeddedPostgres.<init>(EmbeddedPostgres.java:141)
at com.opentable.db.postgres.embedded.EmbeddedPostgres$Builder.start(EmbeddedPostgres.java:545)
at com.opentable.db.postgres.embedded.EmbeddedPostgres.start(EmbeddedPostgres.java:445)
at com.opentable.db.postgres.junit.SingleInstancePostgresRule.before(SingleInstancePostgresRule.java:35)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:41)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
The unit test is copied from from your test:
package com.billerica.manager.database;
import com.opentable.db.postgres.junit.EmbeddedPostgresRules;
import com.opentable.db.postgres.junit.SingleInstancePostgresRule;
import org.junit.Rule;
import org.junit.Test;
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.Statement;
import static org.junit.Assert.*;
public class testSingleInstanceEmbeddedPg {
@Rule
public SingleInstancePostgresRule epg = EmbeddedPostgresRules.singleInstance();
@Test
public void testSingleInstance() throws Exception {
try (Connection c = epg.getEmbeddedPostgres().getPostgresDatabase().getConnection()) {
assertNotNull(epg);
Statement s = c.createStatement();
ResultSet rs = s.executeQuery("SELECT 1");
assertTrue(rs.next());
assertEquals(1, rs.getInt(1));
assertFalse(rs.next());
}
}
}
I see that the temp folder epg3467157523703904606 is empty, if that says anything. I haven't found any log files in the temporary folders.
Other tests give similar results. Sorry if I have missed something obvious; I haven't done any other configuration or setup.
Thanks for any help you can provide.
Did postgres print anything to standard error before failing? initdb should have emit some output and you need to see what that says to understand why it's failing.
Hi Steven, thanks for the reply.
I've been unable to find any sign of logging other than the stack trace (I was running from an IntelliJ IDEA terminal)
However, quite strangely the tests are now passing successfully. So something in my environment has changed to make it work. Previously I certainly had tried a number a times (including shutting down IDEA and rebuilding from scratch.
Something I did do in the meantime, was that I did install Postgres (and drivers for it) on the windows box. Could that have made the difference? My understanding of otj-pg-embedded was that it comes complete with a bundled postgres. Was I wrong in my understanding?
In any case, since things are now functioning (and I can no longer reproduce the problem) I guess we can close this ticket out. I would be interested in any comments you might have on what could have caused this, although there is very little to go on. Sorry for your time.
I have the same issue locally.
My environment:
- MS Windows 10 (build 1607 with all the updates)
- JDK: Oracle HotSpot 1.8.0_161
- Apache Maven: 3.5.2 (the issue also occurs when running the test without Maven i.e. directly from IDE)
- IDE: IntelliJ IDEA Ultimate IntelliJ IDEA 2017.3.4 (Ultimate Edition) Build #IU-173.4548.28, built on January 29, 2018
- Postgresql-embedded: 2.6
Neither setting all logs to DEBUG/TRACE level or debugging helped me.
Running everything as administrator does not change anything.
Any idea?
I have observed this issue while using otj-pg-embedded 0.11.0.
Unfortunately, it still occurs while running 0.11.1 release.
@stevenschlansker
Any idea?
Since none of us at OT currently work with Windows, we are happy to entertain PR's, but can't really resolve it (assuming it's windows specific)
What would be really great is if one of you would run initdb manually on a new postgres install
I do not run PostgreSQL as a separate Windows process.
In this scenario I used Embedded PostgreSQL:
https://github.com/yandex-qatools/postgresql-embedded
Embedded PostgreSQL 2.6 (latest release at the time of writing) runs PostgreSQL 9.6.7 in-memory.
Running initdb.exe manually provides no output:
`
D:\git>C:\Users\krzwil\AppData\Local\Temp\embedded-pg\PG-20610842f7206a9abbecdcf12709c0f8\bin\initdb.exe -A, trust -U postgres -D C:\Users\krzwil\AppData\Local\Temp\epg4200883402029108528 -E UTF-8
`
Is there a way to enforce debug/trace messages on initdb.exe?
Just as a quick update. I have not seen a recurrence of this issue, and cannot reproduce it. Also, a colleague here at work (who did not have PostgreSQL previously installed, and is running with a similar environment to mine) also does not hit this problem while running tests using embedded postgres.
I have tried to more things:
- removing files from C:\Users\krzwil\AppData\Local\Temp\ before running tests
- upgrading to postgresql-embedded from 2.6 to 2.8
None of them changed reported issue a bit.
Any idea?
I have also tried running all tests ('mvn test') on freshly cloned postgresql-embedded master branch.
They failed:
`
T E S T S
Running ru.yandex.qatools.embed.postgresql.config.StorageTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.443 sec
Running ru.yandex.qatools.embed.postgresql.EmbeddedPostgresTest
Extract C:\Users\krzwil.embedpostgresql\postgresql-10.2-1-windows-x64-binaries.zip START
...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Extract C:\Users\krzwil.embedpostgresql\postgresql-10.2-1-windows-x64-binaries.zip DONE
2018-03-05 10:15:37 DEBUG ProcessControl:117 - Detected pid: 11616
2018-03-05 10:15:52 INFO Executable:109 - start AbstractPostgresConfig{storage=Storage{dbDir=C:\Users\krzwil\AppData\Local\Temp\postgresql-embed-4a5ac605-90cc-4596-9591-ecfc6bd8a629\db-content-d5265410-a6dc-4c47-a6c4-ef34e960f92b, dbName='postgres', isTmpDir=true}, network=Net{host='localhost', port=3725}, timeout=Timeout{startupTimeout=15000}, credentials=Credentials{username='postgres', password='postgres'}, args=[], additionalInitDbParams=[-E, SQL_ASCII, --locale=C, --lc-collate=C, --lc-ctype=C]}
2018-03-05 10:15:54 WARN PostgresProcess:122 - Possibly failed to run initdb:
2018-03-05 10:15:54 DEBUG ProcessControl:117 - Detected pid: 12292
2018-03-05 10:16:14 ERROR PostgresProcess:257 - Failed to read PID file (File 'C:\Users\krzwil\AppData\Local\Temp\postgresql-embed-4a5ac605-90cc-4596-9591-ecfc6bd8a629\db-content-d5265410-a6dc-4c47-a6c4-ef34e960f92b\postmaster.pid' does not exist)
java.io.FileNotFoundException: File 'C:\Users\krzwil\AppData\Local\Temp\postgresql-embed-4a5ac605-90cc-4596-9591-ecfc6bd8a629\db-content-d5265410-a6dc-4c47-a6c4-ef34e960f92b\postmaster.pid' does not exist
at org.apache.commons.io.FileUtils.openInputStream(FileUtils.java:299)[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.readLines(FileUtils.java:1783)[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.readLines(FileUtils.java:1819)[commons-io-2.4.jar:2.4]
at ru.yandex.qatools.embed.postgresql.PostgresProcess.onAfterProcessStart(PostgresProcess.java:255)[file:/D:/tmp/postgresql-embedded/target/classes/:]
at de.flapdoodle.embed.process.runtime.AbstractProcess.(AbstractProcess.java:116)[de.flapdoodle.embed.process-2.0.2.jar:]
at ru.yandex.qatools.embed.postgresql.AbstractPGProcess.(AbstractPGProcess.java:19)[file:/D:/tmp/postgresql-embedded/target/classes/:]
at ru.yandex.qatools.embed.postgresql.PostgresProcess.(PostgresProcess.java:68)[file:/D:/tmp/postgresql-embedded/target/classes/:]
at ru.yandex.qatools.embed.postgresql.PostgresExecutable.start(PostgresExecutable.java:25)[file:/D:/tmp/postgresql-embedded/target/classes/:]
at ru.yandex.qatools.embed.postgresql.PostgresExecutable.start(PostgresExecutable.java:13)[file:/D:/tmp/postgresql-embedded/target/classes/:]
at de.flapdoodle.embed.process.runtime.Executable.start(Executable.java:108)[de.flapdoodle.embed.process-2.0.2.jar:]
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:173)[file:/D:/tmp/postgresql-embedded/target/classes/:]
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:142)[file:/D:/tmp/postgresql-embedded/target/classes/:]
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:134)[file:/D:/tmp/postgresql-embedded/target/classes/:]
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:130)[file:/D:/tmp/postgresql-embedded/target/classes/:]
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgresTest.itShouldStartWithDefaults(EmbeddedPostgresTest.java:41)[file:/D:/tmp/postgresql-embedded/target/test-classes/:]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)[:1.8.0_161]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)[:1.8.0_161]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.8.0_161]
at java.lang.reflect.Method.invoke(Method.java:498)[:1.8.0_161]
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)[junit-4.12.jar:4.12]
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)[junit-4.12.jar:4.12]
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)[junit-4.12.jar:4.12]
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)[junit-4.12.jar:4.12]
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)[junit-4.12.jar:4.12]
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)[junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)[junit-4.12.jar:4.12]
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)[junit-4.12.jar:4.12]
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)[junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)[junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)[junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)[junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)[junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)[junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)[junit-4.12.jar:4.12]
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)[:1.8.0_161]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)[:1.8.0_161]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.8.0_161]
at java.lang.reflect.Method.invoke(Method.java:498)[:1.8.0_161]
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)[surefire-api-2.12.4.jar:2.12.4]
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)[surefire-booter-2.12.4.jar:2.12.4]
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)[surefire-booter-2.12.4.jar:2.12.4]
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)[surefire-booter-2.12.4.jar:2.12.4]
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)[surefire-booter-2.12.4.jar:2.12.4]
`
Is it an otj-pg-embedded issue or postgresql-embedded issue then?
I can replicate this one too:
(test-system/start-system!) 18-03-21 16:24:56 mll1p05653.local INFO [com.opentable.db.postgres.embedded.EmbeddedPostgres:668] - Detected a Darwin x86_64 system 18-03-21 16:24:56 mll1p05653.local INFO [com.opentable.db.postgres.embedded.EmbeddedPostgres:707] - Extracting Postgres... 18-03-21 16:25:01 mll1p05653.local INFO [com.opentable.db.postgres.embedded.EmbeddedPostgres:742] - Postgres binaries at /var/folders/_s/zh8x143s1dbf2m0b076b9st1b7y13m/T/embedded-pg/PG-1576ace94f039fde0d83e95305e0acc5 IllegalStateException Process [/var/folders/_s/zh8x143s1dbf2m0b076b9st1b7y13m/T/embedded-pg/PG-1576ace94f039fde0d83e95305e0acc5/bin/initdb, -A, trust, -U, postgres, -D, /var/folders/_s/zh8x143s1dbf2m0b076b9st1b7y13m/T/epg570193259857342578, -E, UTF-8] failed com.opentable.db.postgres.embedded.EmbeddedPostgres.system (EmbeddedPostgres.java:558) user>
@alexanderjamesking thanks for confirming that someone else observes the same issues as me
To sum up, I have checked the following environments:
- MS Windows 10, version 1607 (w/ corporate policies running on a business laptop) - FAILS
- MS Windows 10, version 1703 (vanilla running in VMware machine on a private laptop) - WORKS
- MS Windows 10, version 1709 (vanilla running in VMware machine on a private laptop) - WORKS
- Maven as Docker container https://hub.docker.com/_/maven/ (root user inside Docker container) - FAILS
hi guys, i had the same issue, but when i run initdb.exe manually - widows alert me that some dll is not found, so just download it and it should help
Cheers
Thanks @Egoruch for your piece of advice.
I have installed MS Visual C++ Redistributable Packages for Visual Studio 2013 x64 and it helped me.
I had already installed MS Visual C++ Redistributable Packages for Visual Studio 2013 x86 but it turned out not to be sufficient.
Makes sense! Thanks! We will add to readme then close this
Closing, since the README is now updated.
Had the same issue, even after I had installed Microsoft Visual C++ Redistributable for Visual Studio 2017 (64 bit).
What helped was to install MS Visual C++ Redistributable Packages for Visual Studio 2013 x64 :-)
Thanks!
Right - these are static - and depend on what Postgres was compiled on at the time
I had different error message. I had run time exception stating that inistdb.exe was not found under \user\AppData\Local\Temp\embedded-pg\PG-20610842f7206a9abbecdcf12709c0f8\bin\
This happened without any changes from except I added extra test, which threw the above exception. Then all other tests, which used to work just fine failed. Even another Application on the same machine threw the same exception.
I downloaded MS Visual C++ Redistributable Packages for Visual Studio 2013 x64. I did not help. I was using 0.11.0. I upgraded to 0.12.0, and they all worked. Not sure what happened all of a sudden!
I got the same error in a Mac machine, but seems that the problems was the user's permissions. I've tried execute the command manually and I got an error complaining about this.
command executed:
/var/folders/qc/q9c34bpj3k95fx2vfm6c7jx80000gp/T/embedded-pg/PG-d6f2320465793366f452337868c0d06f/bin/initdb -A trust -U postgres -D /var/folders/qc/q9c34bpj3k95fx2vfm6c7jx80000gp/T/epg8438268694472155640 -E UTF-8After debugging, and running the initdb command directly on the docker agent:
initdb: encoding mismatch The encoding you selected (UTF8) and the encoding that the selected locale uses (LATIN1) do not match.
This means that the locale wasn't set when embedded-postgres db was created. For me it worked by setting the locale:
export LANG=en_US.UTF-8
I faced this issue on mac and seems like it was a permission issue on intellij not being able to run commands. I was successfully able to run test case from terminal using gradle.
I've run into this issue on a Mac too. I tried removing various temp files, but I couldn't make it start correctly. Running the commands directly in my terminal was fine, but not from the JVM. After restarting the computer it ran fine. Not a satisfying result obviously, I'll do some more digging the next time this issue comes up.
Running into this on Mac too. Similar to @danielcompton . Running my test with the IntelliJ runner I get one more clue:
INFO init-190fff25-b380-41d7-af41-01fa8364adeb:initdb - The files belonging to this database system will be owned by user "myuser". [# 17:26:28.193] [log:pid(36102)]
INFO init-190fff25-b380-41d7-af41-01fa8364adeb:initdb - This user must also own the server process. [# 17:26:28.193] [log:pid(36102)]
I started receiving this exception after switching images on our CI to adopopenjdk 11. After running the initdb command from inside the container, I received this output:
/tmp/embedded-pg/PG-06e3a92a2edb6ddd6dbdf5602d0252ca/bin/initdb -A trust -U postgres -D /tmp/epg7204157023524008314 -E UTF-8
initdb: cannot be run as root
Please log in (using, e.g., "su") as the (unprivileged) user that will
own the server process.
Switching the container user from root to circleci fixed this.
I had the same problem and solved it. The exception was the following:
java.lang.IllegalStateException: Process [C:\Users\***\AppData\Local\Temp\embedded-pg\PG-82d8fdc4b690082b3ac0667aee5358bb\bin\initdb.exe, -A, trust, -U, postgres, -D, C:\Users\***\AppData\Local\Temp\epg6545158044000690790, -E, UTF-8] failed
From my spring test the following internally called command failed:
C:\Users\***\AppData\Local\Temp\embedded-pg\PG-82d8fdc4b690082b3ac0667aee5358bb\bin\initdb.exe -A trust -U postgres -D C:\Users\***\AppData\Local\Temp\epg6545158044000690790 -E UTF-8
Doing this out of the windows console worked properly (after adding the C:\Users\***\AppData\Local\Temp\embedded-pg\PG-82d8fdc4b690082b3ac0667aee5358bb\lib folder to the PATH environment variable to prevent the .dll message) and everything was created in C:\Users\***\AppData\Local\Temp\epg6545158044000690790.
Restarting the whole PC and cleaning the C:\Users\***\AppData\Local\Temp folder solved
the problem.
I had the same problem on a Mac, but I was able to solve quite simply.
10:01:20.447 [DEBUG] [TestEventLogger] nyomio.commons.EmbeddedPostgresTest > testEmbedded() STANDARD_OUT
10:01:20.447 [DEBUG] [TestEventLogger] 10:01:20.442 [Test worker] INFO com.opentable.db.postgres.embedded.EmbeddedPostgres - Detected a Darwin x86_64 system
10:01:20.557 [DEBUG] [TestEventLogger] 10:01:20.557 [Test worker] INFO com.opentable.db.postgres.embedded.EmbeddedPostgres - Postgres binaries at /var/folders/4q/***/T/embedded-pg/PG-73dc0043fe7bdb624d5e8726bc457b7e
10:01:20.558 [DEBUG] [TestEventLogger] 10:01:20.558 [Test worker] DEBUG com.opentable.db.postgres.embedded.EmbeddedPostgres - 788f627d-e723-40a9-980a-f7b202965bb7 postgres: data directory is /var/folders/4q/***/T/epg7982076833040837593, postgres directory is /var/folders/4q/***/T/embedded-pg/PG-73dc0043fe7bdb624d5e8726bc457b7e
10:01:20.650 [DEBUG] [TestEventLogger] 10:01:20.650 [log:pid(7092)] INFO init-788f627d-e723-40a9-980a-f7b202965bb7:initdb - The files belonging to this database system will be owned by user "istvan".
10:01:20.650 [DEBUG] [TestEventLogger] 10:01:20.650 [log:pid(7092)] INFO init-788f627d-e723-40a9-980a-f7b202965bb7:initdb - This user must also own the server process.
10:01:20.650 [DEBUG] [TestEventLogger] 10:01:20.650 [log:pid(7092)] INFO init-788f627d-e723-40a9-980a-f7b202965bb7:initdb -
10:01:20.650 [DEBUG] [TestEventLogger] 10:01:20.650 [log:pid(7092)] INFO init-788f627d-e723-40a9-980a-f7b202965bb7:initdb - initdb: invalid locale settings; check LANG and LC_* environment variables
10:01:20.671 [DEBUG] [TestEventLogger]
10:01:20.671 [DEBUG] [TestEventLogger] nyomio.commons.EmbeddedPostgresTest > testEmbedded() FAILED
10:01:20.672 [DEBUG] [TestEventLogger] java.lang.IllegalStateException: Process [/var/folders/4q/zq9fy2hs1kn3xcyt333x_l240000gn/T/embedded-pg/PG-***/bin/initdb, -A, trust, -U, postgres, -D, /var/folders/4q/***/T/epg7982076833040837593, -E, UTF-8] failed
10:01:20.672 [DEBUG] [TestEventLogger] at com.opentable.db.postgres.embedded.EmbeddedPostgres.system(EmbeddedPostgres.java:602)
10:01:20.673 [DEBUG] [TestEventLogger] at com.opentable.db.postgres.embedded.EmbeddedPostgres.initdb(EmbeddedPostgres.java:221)
10:01:20.673 [DEBUG] [TestEventLogger] at com.opentable.db.postgres.embedded.EmbeddedPostgres.<init>(EmbeddedPostgres.java:142)
10:01:20.673 [DEBUG] [TestEventLogger] at com.opentable.db.postgres.embedded.EmbeddedPostgres$Builder.start(EmbeddedPostgres.java:554)
10:01:20.673 [DEBUG] [TestEventLogger] at com.opentable.db.postgres.embedded.EmbeddedPostgres.start(EmbeddedPostgres.java:438)
10:01:20.673 [DEBUG] [TestEventLogger] at nyomio.commons.EmbeddedPostgresTest.testEmbedded(EmbeddedPostgresTest.kt:13)
The line initdb: invalid locale settings; check LANG and LC_* environment was the hint.
You do have to set your localization environment variables to UTF8 like this:
export LC_ALL="en_US.UTF-8"
export LC_CTYPE="en_US.UTF-8"
(according to this stackoverflow answer.)
Of course, you can set these environment variables in your intelliJ runtime config.
Or even better, if you use gradle, you can add environment variables to the test task like this:
test {
useJUnitPlatform()
environment "LC_CTYPE", "en_US.UTF-8"
environment "LC_ALL", "en_US.UTF-8"
}
I cannot try this on windows, but it might help there, too. As far as I know, on windows you have to set the LANG environment variable.
Here like to share my problem and solution.
1> On Mac, using otj-pg-embedded-0.13.1 and 0.13.3. It worked before, but not any more (being long time, there were many different updates onto my Mac).
2> Errors (same from shell command line and Intelij):
java.lang.IllegalStateException:
Process [/var/folders/lj/vg60qx992j5frpmgbs498w940000gp/T/embedded-pg/PG-73dc0043fe7bdb624d5e8726bc457b7e/bin/initdb, -A, trust, -U, postgres, -D, /var/folders/lj/vg60qx992j5frpmgbs498w940000gp/T/epg1897529519870188961, -E, UTF-8] failed
3> Tried suggestions listed in this ticket and had no luck.
4> Identify the root issue and solution:
4.a> run following from shell command (copy/pasted from error above) to reveal the root problem: the openssl was the problem in my case:
$ /var/folders/lj/vg60qx992j5frpmgbs498w940000gp/T/embedded-pg/PG-3d7ce5d05cd575a649dd635576931b19/bin/initdb
dyld: Library not loaded: /usr/local/opt/openssl/lib/libssl.1.0.0.dylib
Referenced from: /usr/local/Cellar/postgresql/11.4/lib/libpq.5.11.dylib
Reason: image not found
Abort trap: 6
4.b> Solution:
-- I have several versions of openssl installed, and a newer version openssl, 1.1.1d, is in use.
-- Solution: install or link to proper openssl version (in my case, 1.0.2s), which includes the proper libssl version being expected by postgresql (in my case, libssl.1.0.0.dylib).
4.c> Both command line and inteliJ work fine afterwards ;-)
5> The bottom line: some other programs may expect a newer/different version of openssl. May have to switch back and forth since i made the changes at system level.
FYI for future people if you use mac and use brew they upgraded it automatically so force downgrading openssl worked in my case as well as @wtangv
I'm having the same issue on Windows. I've verified that I have all of the mentioned MS VS C++ 2013 x64 and that running initdb manually works fine.
Not sure what else to try.
running manually on my mac i get the following error:
dyld: Library not loaded: /usr/local/opt/openssl/lib/libssl.1.0.0.dylib Referenced from: /usr/local/lib/libpq.5.dylib Reason: image not found Abort trap: 6
Running brew update && brew upgrade fixed the issue if you're a mac user with homebrew.
Setting the locale environment variables worked for me.
export LC_ALL="en_US.UTF-8"
export LC_CTYPE="en_US.UTF-8"
In macOS Catalina, set above variables in ~/etc/profile and restart your editor/open new shell.
https://stackoverflow.com/a/8161863/9878734
Hello, I'm having this issue (Process [/tmp/embedded-pg/PG-6e6cb600c5af6f0b241f84df1594a9c6/bin/initdb, -A, trust, -U, postgres, -D, /tmp/epg924981102176549599, -E, UTF-8] failed
) when running my tests on bitbucket pipelines (I think it is using docker). I tried setting the locale environment variables but it didn't work.
Any ideas on how to fix it ?
Thx
use a user that does not have root permission to execute embedded postgres. For some reason user with root access have problems during execution.
example to use in docker
##To change user to a non root user
RUN groupadd -r nonroot && useradd --no-log-init -r -m -g nonroot nonroot && chown -R nonroot:nonroot /src
# this will set the user to run the next commands
USER nonroot
thanks @salluspeaks for your reply! I'll test and let you know
Had the same issue, even after I had installed Microsoft Visual C++ Redistributable for Visual Studio 2017 (64 bit).
What helped was to install MS Visual C++ Redistributable Packages for Visual Studio 2013 x64 :-)
Thanks!
Work for me thanks !!
@dubreuil534 Thanks. MS Visual C++ Redistributable Packages for Visual Studio 2013 x64 worked for me too!
Is there any specific version of redistributable c++ package..as for me nothing worked...changing encoding to utf8 of maven or cleaning temp or installing redistributable package
I had a similar problem running tests with embedded postgres on MacOS M1:
java.lang.IllegalStateException: Process [/var/folders/g_/jxzv9glj0p13njlzpr9xtcfm0000gn/T/embedded-pg/PG-4f38c8a8b2500189835f8ec1b75de81b/bin/initdb, -A, trust, -U, postgres, -D, /var/folders/g_/jxzv9glj0p13njlzpr9xtcfm0000gn/T/pg-embedded-009358be-3e95-4dbe-aa7a-01a9008511cb-3277843270072134584/epg524445573999525527, -E, UTF-8] failed
I copy pasted the command and run it on shell:
/var/folders/g_/jxzv9glj0p13njlzpr9xtcfm0000gn/T/embedded-pg/PG-4f38c8a8b2500189835f8ec1b75de81b/bin/initdb -A trust -U postgres -D /var/folders/g_/jxzv9glj0p13njlzpr9xtcfm0000gn/T/pg-embedded-009358be-3e95-4dbe-aa7a-01a9008511cb-3277843270072134584/epg524445573999525527 -E UTF-8
and the output show me a hint:
Running bootstrap script ... 2021-02-17 08:18:21.008 WET [28692] FATAL: could not create shared memory segment: Cannot allocate memory
2021-02-17 08:18:21.008 WET [28692] DETAIL: Failed system call was shmget(key=5432001, size=56, 03600).
2021-02-17 08:18:21.008 WET [28692] HINT: This error usually means that PostgreSQL's request for a shared memory segment exceeded your kernel's SHMALL parameter. You might need to reconfigure the kernel with larger SHMALL.
The PostgreSQL documentation contains more information about shared memory configuration.
child process exited with exit code 1
initdb: removing contents of data directory "/var/folders/g_/jxzv9glj0p13njlzpr9xtcfm0000gn/T/pg-embedded-cbe6f5f8-24b2-4431-afec-6e7aeb1dbb5d-8656684715588054403/epg6904898061245880853"
Then I run on shell:
sudo sysctl kern.sysv.shmmax=104857600
sudo sysctl kern.sysv.shmall=25600
And it worked.
Thanks
I had a similar problem running tests with embedded postgres on MacOS M1:
java.lang.IllegalStateException: Process [/var/folders/g_/jxzv9glj0p13njlzpr9xtcfm0000gn/T/embedded-pg/PG-4f38c8a8b2500189835f8ec1b75de81b/bin/initdb, -A, trust, -U, postgres, -D, /var/folders/g_/jxzv9glj0p13njlzpr9xtcfm0000gn/T/pg-embedded-009358be-3e95-4dbe-aa7a-01a9008511cb-3277843270072134584/epg524445573999525527, -E, UTF-8] failedI copy pasted the command and run it on shell:
/var/folders/g_/jxzv9glj0p13njlzpr9xtcfm0000gn/T/embedded-pg/PG-4f38c8a8b2500189835f8ec1b75de81b/bin/initdb -A trust -U postgres -D /var/folders/g_/jxzv9glj0p13njlzpr9xtcfm0000gn/T/pg-embedded-009358be-3e95-4dbe-aa7a-01a9008511cb-3277843270072134584/epg524445573999525527 -E UTF-8and the output show me a hint:
Running bootstrap script ... 2021-02-17 08:18:21.008 WET [28692] FATAL: could not create shared memory segment: Cannot allocate memory 2021-02-17 08:18:21.008 WET [28692] DETAIL: Failed system call was shmget(key=5432001, size=56, 03600). 2021-02-17 08:18:21.008 WET [28692] HINT: This error usually means that PostgreSQL's request for a shared memory segment exceeded your kernel's SHMALL parameter. You might need to reconfigure the kernel with larger SHMALL. The PostgreSQL documentation contains more information about shared memory configuration. child process exited with exit code 1 initdb: removing contents of data directory "/var/folders/g_/jxzv9glj0p13njlzpr9xtcfm0000gn/T/pg-embedded-cbe6f5f8-24b2-4431-afec-6e7aeb1dbb5d-8656684715588054403/epg6904898061245880853"Then I run on shell:
sudo sysctl kern.sysv.shmmax=104857600 sudo sysctl kern.sysv.shmall=25600And it worked.
Thanks...but for me it's Windows.
I faced this issue on mac and seems like it was a permission issue on intellij not being able to run commands. I was successfully able to run test case from terminal using gradle.
How do you fix it? I faced with the same propblem
10:01:20.447 [DEBUG] [TestEventLogger] nyomio.commons.EmbeddedPostgresTest > testEmbedded() STANDARD_OUT 10:01:20.447 [DEBUG] [TestEventLogger] 10:01:20.442 [Test worker] INFO com.opentable.db.postgres.embedded.EmbeddedPostgres - Detected a Darwin x86_64 system 10:01:20.557 [DEBUG] [TestEventLogger] 10:01:20.557 [Test worker] INFO com.opentable.db.postgres.embedded.EmbeddedPostgres - Postgres binaries at /var/folders/4q/***/T/embedded-pg/PG-73dc0043fe7bdb624d5e8726bc457b7e 10:01:20.558 [DEBUG] [TestEventLogger] 10:01:20.558 [Test worker] DEBUG com.opentable.db.postgres.embedded.EmbeddedPostgres - 788f627d-e723-40a9-980a-f7b202965bb7 postgres: data directory is /var/folders/4q/***/T/epg7982076833040837593, postgres directory is /var/folders/4q/***/T/embedded-pg/PG-73dc0043fe7bdb624d5e8726bc457b7e 10:01:20.650 [DEBUG] [TestEventLogger] 10:01:20.650 [log:pid(7092)] INFO init-788f627d-e723-40a9-980a-f7b202965bb7:initdb - The files belonging to this database system will be owned by user "istvan". 10:01:20.650 [DEBUG] [TestEventLogger] 10:01:20.650 [log:pid(7092)] INFO init-788f627d-e723-40a9-980a-f7b202965bb7:initdb - This user must also own the server process. 10:01:20.650 [DEBUG] [TestEventLogger] 10:01:20.650 [log:pid(7092)] INFO init-788f627d-e723-40a9-980a-f7b202965bb7:initdb - 10:01:20.650 [DEBUG] [TestEventLogger] 10:01:20.650 [log:pid(7092)] INFO init-788f627d-e723-40a9-980a-f7b202965bb7:initdb - initdb: invalid locale settings; check LANG and LC_* environment variables 10:01:20.671 [DEBUG] [TestEventLogger] 10:01:20.671 [DEBUG] [TestEventLogger] nyomio.commons.EmbeddedPostgresTest > testEmbedded() FAILED 10:01:20.672 [DEBUG] [TestEventLogger] java.lang.IllegalStateException: Process [/var/folders/4q/zq9fy2hs1kn3xcyt333x_l240000gn/T/embedded-pg/PG-***/bin/initdb, -A, trust, -U, postgres, -D, /var/folders/4q/***/T/epg7982076833040837593, -E, UTF-8] failed 10:01:20.672 [DEBUG] [TestEventLogger] at com.opentable.db.postgres.embedded.EmbeddedPostgres.system(EmbeddedPostgres.java:602) 10:01:20.673 [DEBUG] [TestEventLogger] at com.opentable.db.postgres.embedded.EmbeddedPostgres.initdb(EmbeddedPostgres.java:221) 10:01:20.673 [DEBUG] [TestEventLogger] at com.opentable.db.postgres.embedded.EmbeddedPostgres.<init>(EmbeddedPostgres.java:142) 10:01:20.673 [DEBUG] [TestEventLogger] at com.opentable.db.postgres.embedded.EmbeddedPostgres$Builder.start(EmbeddedPostgres.java:554) 10:01:20.673 [DEBUG] [TestEventLogger] at com.opentable.db.postgres.embedded.EmbeddedPostgres.start(EmbeddedPostgres.java:438) 10:01:20.673 [DEBUG] [TestEventLogger] at nyomio.commons.EmbeddedPostgresTest.testEmbedded(EmbeddedPostgresTest.kt:13)
@istvanszoboszlai Any chance you remember how you got such a neat output ? We're having issues debugging a gitlab CI pipeline and getting the output of the initdb command seems key to understanding what is missing.
Updating brew fixed the issue brew update && brew upgrade
Thanks pekevski
For people who are using PostgreSQL >= 15 under Windows 10+ you might need to install Visual C++ Redistributable for Visual Studio 2015. At least for me 2013 did not solve the problem.