Jarkko's How to build Configure tweaked by Nick and Merijn, and now
maintained by perl5-metaconfig
The Configure script and config_h.SH file in the Perl distribution are
generated by a program called metaconfig. Metaconfig was originally
written by Larry Wall, and was subsequently enhanced and maintained
by Raphael Manfredi. The binary that invokes the generation of the
Configure file is called mconfig.
As sort order and filenaming are vital in this process, make sure you
are working on a case-sensitive file system! (Case preserving is not
sufficient).
You have presumably obtained the metaconfig from the repository e.g.
$ git clone git@github.com:Perl/metaconfig metaconfig
When working with metaconfig you will generally have two git checkouts
next to each other: (1) this metaconfig checkout; and (2) a checkout of
the Perl source code in which you will generate a new Configure script. In this
README, we will refer to these directories as the 'metaconfig' directory and
the 'perl' directory.
Since these two directories are normally next to each other, so ../perl
will get you to perl and ../perl/../metaconfig will get you back here.
You should establish a symbolic link to the checkout in which Configure
is generated such as this:
$ cd metaconfig
$ ln -s ../perl perl
We will do the reverse symlinks later.
Contents of this directory:
README: This file.
U: Metaconfig units used for building Perl's Configure
dist-git:
a git clone of "dist". Optionally present. See (a) below.
This is where dist/meta resides as of 2016-04-01
dist:
The folder where the original units from dist are in.
These may differ from dist-git, as upstream also moves
on and develops.
Development workflow:
(a) In order to assemble Configure from its units, you need mlint/metalint and
mconfig/metaconfig from the "dist" package installed and available in your
$PATH. You can either use the version that comes with your OS (Debian ships
it) or the versions that are included in this checkout: just add the full
name of this folder/bin to your $PATH. If you are not planning to analyse
differences of the current state with upstream dist, you can skip the rest
of step (a) now.
If you also want to play with or compare to the original meta/dist, you
can checkout that too.
The dist version used for perl is dist-3.5-20 in this directory, which is
a slightly modified version of the original, which you can get at GITHUB
repository https://github.com/rmanfredi/dist.git. If you'd like to keep
up to date with changes in dist, you can use git to create your own clone.
For git, that would be something like:
$ git clone https://github.com/rmanfredi/dist.git dist-git
Unsurprisingly 'dist' uses (its) Configure to generate itself:
$ cd dist-3.5-20 # or dist-git
$ chmod -R +w . # We have derived files in git :-(
$ ./Configure
$ make
$ make install
After make install, remove lib/U/d_debugging.U in your target lib, as perl
uses its own way to set/define debugging (see INSTALL)
dist's 'Configure' is similar to perl's but perhaps not quite as polished.
There are some perl specific "dist units" in the 'U' directory.
The U directory also contains some patches to 'dist' which have already
been applied to dist-3.5-20 directory.
(aa) We have not yet arranged for metaconfig to use perl's versions of the
'units' by default so you need some housekeeping in the perl directory...
Add metaconfig/bin to your $PATH or create aliases like
$ export MC5=/your/path/to/metaconfig
$ alias ml="perl $MC5/bin/mlint -O"
$ alias mc="perl $MC5/bin/mconfig -m -O"
examples in the rest of this README will just refer to mlint and mconfig
as if they appear in your $PATH
(aaa)
If you plan to make changes to mconfig or mlint locally (and you might
want to, as both are written for perl4), consider installing mconfig and
mlint from the cmon subdirectory into your $PATH too. These are the
non-autoloading versions and can easily be changed. As these are used by
all team members, please communicate changes on github first.
(b) You need to be in the 'perl' checkout directory, which you created the
symbolic link to, in preparation. In this working directory, you need
symbolic links too, which are already known to perl itself to ignore.
Assuming you have metaconfig and perl side by side on the same level:
ln -s ../metaconfig/U U
ln -s ../metaconfig/.package .package
ln -s MANIFEST MANIFEST.new
chmod +w Configure config_h.SH Porting/Glossary Porting/config*
(c) Create a new file for the unit as U/foo/d_bar.U
('foo' is one of the existing folders in U except for 'all'. If you are
modifying a unit already in dist, simply copy the dist version to
'modified' as a starting point. Otherwise, create a new file in one of the
other directories. It most likely will be 'perl', but it could also be
'compline' or any other existing folder). Choose the best appropriate
subdir of U. See U/README for a description of the various subdirectories.
You should choose the closest existing unit file as a starting point, and
first copy it to the new file. For example, the unit for seeing if
strtold_l() exists was created as U/threads/d_strtold_l.U, copied from
perl/d_strtold.U, then adjusted. It goes under 'threads' because it is
used only on threaded perls.
(d) Run "mlint -O" to see nits: as opposed to lint, the gripings of mlint
are usually serious and need fixing
Without -O, exceptions are lots of
Your private U/modified/issymlink.U overrides the public one.
due to the perl special units
and
"End.U": stale ?MAKE: dependency '$W'.
which is apparently normal ...
-- the next steps are in the perl folder, though the instructions below include
a 'cd perl' at each step, as a reminder. If you already are in 'perl',
disregard the reminders.
(e) There is a chicken and egg problem for newly created units. To get around
this, for such a unit, edit the file metaconfig.h and add to the comment
the appropropriate name. To continue the example above, we would add the
string HAS_STRTOLD_L at the end of the comment. This can be removed once
the code base has actual uses of the unit.
(f) "mconfig -m -O" to regenerate Configure and config_h.SH
Make *sure* your mconfig is the correct one in your $PATH, as the mono-web
package will install /usr/bin/mconfig which will do something completely
different.
(g) metaconfig does not deal with depends in config_h.SH, so some
reorganization is needed.
$ cd perl
$ perl Porting/config_h.pl
will fix the ordering
(h) The messy semi-automated part is that the knowledge of the new symbol
needs to be propagated to non-Configure lands like Win32, WinCE, Netware,
VMS, VOS, ... see previous Configure changes to see which are these
heathen lands. Files to take care of are
{win32,wince,NetWare}/config_[hH]*, (Win32, WinCE, NetWare),
configure.com (VMS). Depending on the kind of patch djgpp/config* might
also need adjusting (for example when adding/changing the list of
extensions)
Most can be checked and updated by a tool Nicholas provided:
$ cd perl
$ perl Porting/checkcfgvar.pl
and if it shows differences, use one of:
$ perl Porting/checkcfgvar.pl --regen --default=undef
$ perl Porting/checkcfgvar.pl --regen --default=define
based on the changes you made. For safety, probes should probably be
'undef', whereas some other things unconditionally should default to
'define'. For example, 'default_inc_excludes_dot' should be 'define'
except in very limited circumstances, because it closes a security hole.
For Win32 the process is semi-automated. You have to have a Win32
machine to run dmake on to complete the process, but that can be done
later by someone with such access.
For VMS, ('configure.com'), you will have to add things manually. It may
be best to add the units as 'undef' and let the VMS experts deal with them
later. However, you can set them to 'define' if they are non-tricky (such
as being basic functions having standard signatures across architectures),
and are in the oldest release of VMS that perl can be compiled on, which is
7.3-2. Appendix A of "HP C Run-Time Library Reference Manual for OpenVMS
Systems" gives you that information. As of October 2017, the latest
version online is available at:
http://h41379.www4.hpe.com/doc/84final/5763/5763profile.html
In configure.com, if there is an existing probe that is essentially the
same (except for the names) as the one you're adding, you can copy, paste,
and adjust to create a new one, but note that it's easy to run afoul of the
quoting rules in configure.com. New probed-for units likely will require
at least 2 groups of changes.
Rerun checkcfgvar.pl until you've fixed everything it finds.
(i) Check if U/mkglossary (right near the top) points to where you keep
dist's standard metaconfig units as well as your perl-specific ones.
(j) Run the perl build chain
$ cd perl
$ make veryclean # Only if Configure already has been run
$ ./Configure -Duse64bitall -Dusethreads -Dusedevel -des
Then make and make test or make test_harness
$ make -j12
$ env TEST_JOBS=13 make test_harness
Before you start committing, make sure that
$ make test_porting
still passes
(k) Optionally, run Porting/mksample to freshen the Porting/config*.
Adjust the various compile-time options (e.g. 64bit, threads) as
you see fit.
You can skip this step, it's not essential, just good housekeeping.
Most of this only works if you have run the core-tests with the new
generated files
(kk) Run U/mkgloss.pl to freshen Porting/Glossary
You should at least check
$ perl U/mkgloss.pl | diff Porting/Glossary -
This will show two warnings that you can ignore:
U/mkglossary: couldn't find libdb_needs_pthread
U/mkglossary: couldn't find libdirs
all other things need a review
-- the next steps are in the metaconfig folder again
(l) git add U/foo/bar.U when you are ready ...
(m) git commit -m "Your commit description"
(n) When all patches are applied, tested and committed, and you are happy,
git push
References:
Documentation on 'dist' may be found at these locations:
https://github.com/rmanfredi/dist/blob/master/mcon/man/mconfig.SH
https://manpages.debian.org/stretch/dist/metaconfig.1.en.html
Git tags:
Tags are maintained in this git repository mapping the version of the
units that were used for the Configure in a given release of perl,
named simply after the version of perl in question (for example, at
the time of writing the current stable release is 5.26.1). This provides
a stable reference for downstreams wishing to import the metaconfig units
into their own packaging. Therefore, at minimum tags for each stable
release should be made (adding tags for development releases being an
optional extra).