| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
| |
- Prevented full ports tree build when operating with ~/.cvsrc
PR: ports/139121
|
|
|
|
|
| |
PR: 138774
Approved by: edwin
|
|
|
|
| |
While here, remove some staleness.
|
|
|
|
|
| |
we create a new src branch. While here, garbage collect some code that
hasn't been run for years.
|
| |
|
| |
|
| |
|
| |
|
| |
|
|
|
|
| |
interest in reviving ia64. While here, generalize things a bit.
|
|
|
|
|
|
|
| |
disks.
While here, note that there is some interest in reviving ia64 package
builds.
|
| |
|
|
|
|
| |
when invoking pdispatch.
|
|
|
|
|
| |
PR: 131440
Requested by: Hannes Hauswedell <freebsd@soulrebel.in-berlin.de>
|
|
|
|
|
|
|
|
| |
available on the ftp mirrors alongside the packages [1]
- While I'm here, remove a NOOP check for FreeBSD 4.x -exp
PR: 135024 [1]
Requested by: Dominic Fandrey <kamikaze@bsdforen.de>
|
| |
|
|
|
|
|
|
|
| |
confused by locally installed ports that might influence
PKGNAME or other variables.
Suggested by: pav
|
| |
|
| |
|
|
|
|
| |
now that it is available in all current builds.
|
|
|
|
|
| |
normal builds instead of a local cvs update. This speeds up the
script several times and easies I/O on pointyhat.
|
| |
|
|
|
|
| |
update from NFS mount, which is just way too slow
|
| |
|
|
|
|
|
| |
o fix rubygem skipping
o next, not die
|
|
|
|
| |
Discussed with: pav on irc (#bsdports)
|
| |
|
|
|
|
| |
instead of a single '_' to follow the current patch naming guidelines.
|
|
|
|
| |
- don't change the Makefile if MAKE_JOBS_(UN)SAFE is already present
|
|
|
|
| |
- Add --ports -p to specify a list on the cli
|
|
|
|
|
|
|
| |
Please be sure to completely test the results of running this _BEFORE_ committing
several times and in a Tinderbox
Feel free to commit to this file
|
| |
|
| |
|
|
|
|
|
| |
- Fix false positives for 'cluster'.
- Add 'error: invalid pure specifier' case to badc++.
|
|
|
|
|
| |
Previously a patch created by update-patches would be changed again
by the next update-patches run.
|
| |
|
|
|
|
| |
Approved by: edwin
|
|
|
|
| |
Submitted by: pgollucci
|
| |
|
|
|
|
|
|
| |
Now re-enabled on production.
Approved by: portmgr (self)
|
|
|
|
|
|
| |
suppress hrefs for non-existant files (incomplete).
Approved by: portmgr (self)
|
|
|
|
| |
Approved by: portmgr (self)
|
|
|
|
|
|
|
|
|
| |
- bring this closer to the default FreeBSD page style
- remove unsupported releases
- remove the date stamps, which no longer work
- remove obsolete commented-out junk
Discussed on: portmgr, some time ago
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
list : lists available builds
clone : creates a new build by cloning a previous one
portsupdate : update a ports tree to the latest ZFS snapshot
srcupdate : update a src tree to the latest ZFS snapshot
cleanup : clean up or remove a build on the clients
destroy : remove a build on the server
There is some trickiness here in that various commands either expect
to run as root, or expect to run as a ports-* user. For the latter
case we can easily use su to proxy as the ports user when running as
root; for the former we use the buildproxy to validate and re-execute
the command as root.
|
|
|
|
| |
keep dozens of full distfile sets lying around.
|
| |
|
|
|
|
| |
expected commands will be proxied)
|
| |
|
|
|
|
|
|
|
|
| |
the ports-* users. Currently it is not possible to delegate
management of ZFS filesystems to non-root users, so root privilege
is required to manipulate them. We validate the command passed on
a local domain socket and re-execute the build script with the requested
parameters.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
ports and source trees. Since we have >=1 consumer of these trees
that run frequently but do not insist on up-to-the-second trees, it
makes sense to "pre-update" them regularly and then then re-use in all
of the consumers, instead of potentially doing several updates
simultaneously or on demand. Consumers can clone the ZFS snapshot
into their local filesystem which takes a couple of seconds instead of
minutes or tens of minutes for the CVS update.
We update to a date stamp instead of "." because this avoids
ambiguity of commits that happen while the tree update is in progress
(unfortunately it's slower).
|
|
|
|
|
|
|
|
| |
Currently we collect:
* The current and maximum number of vnodes in use
* The number of packages built over the past hour
|
| |
|
|
|
|
|
|
| |
listed filesystem we take a new snapshot each time it is run and if
the last full backup was not too long ago, do a compressed incremental
backup from the previous backup.
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Catch up to build ID directory changes
* Support a meta-hostname of 'all' for setting up all clients at once.
This is better than the old way of running one copy of the script
for each client by hand, since it is easier and involves less
duplicated work.
* We copy in the per-build ports, src, and bindist .tbz files and .md5
checksums, as well as refreshing the build scripts and
bindist-$(hostname).tar customization tarball.
* The -force switch forces copying of files and re-extraction of the
tarballs on the client. This is necessary in order to propagate
local changes to the tarballs after the initial client setup
(e.g. if you need to change a file in the ports tree, it must be
recompressed, redistributed, and re-extracted on the client).
* The -queue switch will poll the client's job queue after completion
of the setup. This is racy and should only be used when the machine
is not currently accepting jobs.
* For cleaning up a build the 'build cleanup' command should now be
used instead. It calls back into this command but also allows full
clenaup of build-local files on the client.
TODO: "all" setups are hard on the server since they may spawn dozens
of rsyncs at once. A better solution would be to have a worker pool
of setup tasks to limit the maximum load.
|
|
|
|
|
|
|
|
|
|
| |
* Catch up to build ID directory changes
* Make it easier to kill a build by not running dopackages in the background
where it is detached from shell job control. Now, sending a termination
signal to this process (e.g. ^C) will also kill off the dopackages script
and in turn the processes created by it. Some background processes
spawned by dopackages, pdispatch, etc, may still remain and need to be
killed by hand.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Catch up to build ID directory changes
* Improve usage()
* Fix a variety of small bugs
* Remove support for -ftp builds: we have not supported direct
uploading for many years due to the desire to manually inspect build
output for quality
* All data associated to a build is now localized in its own directory
named according to a build ID:
/var/portbuild/${arch}/${branch}/builds/${buildid}, where ${buildid}
is the creation time. These are actually ZFS filesystems.
* Tasks such as cloning a new build, updating a ZFS snapshot, and
cleaning up a build are exported to the "build" script, which can be
used independently.
* Creating a new build is done by ZFS cloning and takes a couple of
seconds since it is copy-on-write (i.e. no data needs to be copied).
* Ports and source trees are also cloned from pre-updated ZFS images
(updated regularly from the "updatesnap" cron job). In most cases
we do not care if we are building a ports tree that is an hour or so
old since it will become outdated almost immediately anyway, so no
matter what we do there will be times when a port has been fixed by
the time the build error is generated by a client.
* In case an up-to-the-second tree is desired, the -portscvs and
-srccvs switches update the existing ports tree via CVS.
* -noports and -nosrc can be used to prevent any automatic changes to
the ports tree. This is useful for dealing with local
modifications (e.g. for -exp builds), since the default when
creating a new build is to replace the previous trees with fresh,
pristine trees. If you forget to use this then any local changes
that are not also present in other trees will be lost.
* By default we keep two builds for each arch/branch pair. These
build IDs also may be referred to via "latest" and "previous"
symlinks. When creating a new build, the old "previous" build is
destroyed by default, unless it was originally created using the
-keep switch. This prevents the build from being destroyed
automatically.
* By default when a build finishes all of the clients are completely
cleaned up (i.e. all build data such as ports trees, tarballs,
client chroots, etc are deleted). This is needed to save space on
the clients. If you expect to *immediately* perform further builds
after this one completes, the -nocleanup switch prevents this step.
Otherwise they will just be set up again if further builds are
scheduled.
* Try to parallelize build pre-processing as much as possible, by
running jobs in the background wherever possible. In several places
we operate on the same parts of the filesystem from multiple jobs,
so we can make good use of caching to improve performance
* Clients no longer need to be set up explicitly at the start of the
build, they will be set up on-demand when the first job is
dispatched to them. This allows fast clients or those that already
have been set up to begin building ports as soon as possible, while
slow clients are set up in the background. It also improves
robustness of client recovery, e.g. if the client was offline at the
time of build startup but later brought back online.
* Optimize copying back in the previous set of restricted packages by
hardlinking instead of copying.
TODO: The record of failed ports is arch/branch-global still. This is
the only thing preventing us from running concurrent builds of the
same arch/branch (e.g. while one is stuck building openoffice, the
next build can start to keep the cluster busy). The difficulty is
that one build from a later ports tree may signal that a build was
successful, then a phase 2 build from an earlier ports tree may
indicate that it was broken. The solution is probably to migrate this
to a real database instead of a flat file, and query it for the set of
broken ports as of a certain ports tree date.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Clients no longer mount ports/src trees via NFS (even the FreeBSD.org
local clients). This was putting too much load on the server and
slowing down builds.
* Instead ports and src .tbz files are pushed to the clients and
unpacked. MD5 checksums are used to verify correctness
* -force forces re-extraction of the tarballs even if they exist and
appear to be checked out
* Also unpack the compressed bindist
TODO: When we are not using md or ZFS builds it would be even faster
to keep an unpacked copy of the bindist on the scratch filesystem and
hardlink the files into the target directory
|
|
|
|
|
| |
* Optimize by copying old packages using cpio -dumpl (i.e. create hardlink
instead of copying the file).
|
| |
|
|
|
|
|
| |
* Catch up to build ID directory changes
* Remove need for /etc/arch file
|
|
|
|
|
|
| |
* Catch up to build ID directory changes
* Improved support for ZFS
* Desupport X11BASE
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Catch up to build ID directory changes
* Improved support for ZFS builds
* Improved robustness
* Report status verbosely to the caller; whether we succeeded in claiming
a chroot, whether the caller needs to first set up the client, or
whether a setup is in progress.
* If we discover that the client has not been set up either because it
freshly booted and newfs'ed its filesystem, or because a particular
build has not yet been encountered, atomically claim a cookie and
report this to the caller to act on
|
|
|
|
| |
* Catch up to build ID directory changes
|
|
|
|
|
|
|
| |
* Catch up to build ID directory changes
* Add helper functions for resolving a build ID symlink and
validating an arch/branch combination (centralize instead of doing it
in many scripts)
|
|
|
|
| |
* Catch up to build ID directory changes
|
| |
|
| |
|
|
|
|
|
| |
* Catch up to build ID directory changes
* Record package build completion for reporting to ganglia
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Catch up to build ID directory changes
* Add support for ssh_cmd and scp_cmd to allow using HPN-SSH with the
none cipher where possible (for performance)
* Lazy client setup; claim-chroot will report if the client needs to be
set up with this buildid, and we initiate the setup and poll until
it is complete. This allows fast clients to begin building before
slow ones have finished setting up.
TODO: a better solution would be to avoid trying to dispatch jobs onto
clients that are in the process of setting up, since they often have low
loads and are picked preferentially by the job scheduler.
|
|
|
|
|
|
|
|
|
|
| |
* Remove vestiges of archaic support for building bindists from FTP
snapshots; we haven't used this for years and building a world is no
longer a challenge
* Revert half-baked bindist generation number and make it per-buildid
instead. Compress and md5 it for distribution to the clients.
TODO: Merge with makeworld?
|
|
|
|
|
|
| |
* Catch up to build ID directory changes
* Optimize by using ECHO_MSG=true instead of /usr/bin/true
* Try harder to avoid pollution from local host
|
|
|
|
| |
* Catch up to build ID directory layout
|
|
|
|
|
|
| |
* Catch up to build ID directory changes
* Export the INDEX_PRISTINE and INDEX_QUIET variables (old bug)
* Desupport X11BASE
|
|
|
|
|
| |
* Catch up to build ID directory changes
* Desupport 5.x
|
|
|
|
|
| |
* Catch up to build ID directory changes
* Optimize by using __MAKE_SHELL=/rescue/sh
|
|
|
|
|
|
|
|
|
| |
checkmachines script. Polls build machines for their status either
once-off or regularly as a daemon. Optionally it will update the
queue entries but this remains subject to race conditions.
TODO: Integrate with queue manager and forward machine status changes
to it
|
|
|
|
|
| |
report error status, architecture and OS version, and available build
environments, as well as load and number of running jobs
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
targets.
* Use /rescue/sh for index builds instead of /bin/sh, when it exists.
The former is statically linked and faster to execute, which becomes
significant when executing it tens of thousands of times. This
trick can be used with other recursive targets by passing in
__MAKE_SHELL.
* Get rid of make variable assignments that use != command invocations
in the critical path, using several methods:
- rewriting logic to use shell or make builtins instead of external command executions
- macroizing commands and executing them in the targets where they
are needed instead of with every invocation of make
- precomputing the results of invariant commands in
bsd.port.subdir.mk and passing them in explicitly to child makes,
and using this to avoid recalculation in all the children. NB: the
commands are still run one per top-level subdirectory but this
does not currently seem to be a major issue. They could be moved
further up into the top-level Makefile at the cost of some
cleanliness.
- Committers are strongly discouraged from adding further "bare" !=
assignments to the ports tree, even in their own ports. One of
the above strategies should be used to avoid future bloat.
* Rewrite the core 'describe' target to work entirely within a single
shell process using only builtin commands. The old version is
retained as a backup for use on systems older than 603104, which
does not have the make :u modifier. This cuts down the number of
processes executed during the course of a 'make index' by an order
of magnitude, and we are essentially now amortized to the minimum of
a single make + sh instance per port, plus whatever commands the
port makefile itself executes (which are usually unnecessary and
bogus).
* Less validation of the WWW: target is performed; this can become
policed at a port level by portlint. Specifically we look at the
second word of the first line beginning with "WWW:" in pkg-descr,
and append "http://" to it unless it already begins with "http://",
"https://" or "ftp://". Thanks to dougb for the idea of how to
extract WWW: using shell builtins.
* Use the "true" shell builtin instead of echo > /dev/null for a
measurable decrease in CPU use.
* Add a note about dubious escaping strategy in bsd.port.subdir.mk
* Minor change in output of 'make describe': it no longer strips
trailing CR characters from pkg-descr files with MSDOS CR/LF
termination. Instead the makeindex perl script that post-processes
make describe into the INDEX is tweaked to strip on input.
The bottom line is that on my test hardware INDEX builds are now
faster by more than a factor of 2 and with a reduction in system time
by a factor of 4-8 depending on configuration.
|
|
|
|
|
|
|
|
|
|
| |
Because the $FreeBSD$ keyword isn't expanded in the new version, we can't
just do a diff, check the return value and ignore the output.
Every new modules file, changed or not with regarding to the contents,
has at least four lines in the diff output (line number, old line,
seperator, new line). Only commit it if there are more than four
lines difference between it.
|
|
|
|
| |
in the ports CVS repository.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- added -c, doesn't change anything
- added -n, deal with an already checkouted tree
- removed negative logic in favor of 'unless'
- switch to 3 arg form of open()
- don't use globs for filehandles, this is been obsolete
since at least 5.6.1
- handle possible errors in close()
- allow CVSROOT to be overriden in the ENV
PR: ports/125025
Submitted by: "Philip M. Gollucci" <pgollucci@p6m7g8.com>
|
|
|
|
|
|
| |
- no more 5-exp
- add 8, 8-exp
- fix two error-name hrefs
|
| |
|
|
|
|
| |
for this from these scripts.
|
|
|
|
|
|
|
|
|
|
| |
* Remove 5.x support
* Leave the archaic ftp snapshot support for now, it is not hurting anything
but will not work
* Be more careful when removing files (use absolute paths)
* Switch to bindist/tmp for the tmp dir
* Fix the recording of the bindist.tar generation number
* Get rid of redundant or useless processing of the world image
|
|
|
|
|
|
| |
* Record the CVS update stamp in some extra places and make sure to remove it
if the build is started with -noportscvs (since this probably means the
ports tree was updated by hand at some random time)
|
| |
|
|
|
|
| |
them in batches according to their target directory.
|
|
|
|
| |
and make sure they have been post-processed first.
|
|
|
|
|
| |
* Keep RESTRICTED distfiles in a separate DISTDIR so we can easily
avoid accidentally publishing them to the FTP site (idea from des@)
|
|
|
|
| |
Tested on: devel/gettext bump
|
|
|
|
|
|
|
| |
invocations). It also fixes some edge cases that were not handled in
the previous version.
TODO: Correctly report IPv6 sockets (already in use by the sparc64 build)
|
| |
|
|
|
|
| |
Forgotten by: linimon
|
| |
|
|
|
|
| |
header format of the log files.
|
|
|
|
| |
Forgotten by: pav
|
|
|
|
|
| |
INDEX once and process internally instead of invoking many external
utilities, runtime is improved from ~20 minutes to <10 seconds.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
ordering, which had become too limited.
We now build packages ordered by those that are part of the longest
dependency chains first. This has the effect of building the deepest
parts of the tree first and levelling out the tree height, hopefully
avoiding the situation we currently face where there appear
bottlenecks late in the build where the cluster becomes mostly idle
while waiting for a few long dependency chains to finish building
before the cluster can become fully loaded again.
The algorithm is that we sort the list of remaining packages according
to height (longest dependency chain), then add leaf packages from each
in order until we have filled a queue of length between 100 and 200,
to amortise the cost of this queue rebalancing while not losing the
height averaging property. Jobs are dispatched from this queue into
worker threads as machine slots become available.
Unlike the make-based solution that required a fixed -j concurrency
value and could not respond to addition/removal of build resources, we
now can dynamically add new machines as they become available to the
queue.
The other advantage of using python is that we have more
customisability and visibility into the build status, e.g. we
periodically report the number of remaining packages, as well as the
list of deepest packages that we are working on.
TODO:
* Implement mtime checking for parent package staleness, so that
parents are rebuilt if the dependencies are touched more recently.
Currently packages will not be rebuild if they exist, whether or not
they are "stale" wrt their dependencies.
* Offload the machine selection into an external queue manager.
Currently the queue manager used here doesn't interoperate with the
old one (getmachine/releasemachine) because it's not possible to use
the lockf()-based mutual exclusion within a multithreaded client.
Doing that will also allow for a more flexible job placement
algorithm as well as finer queue customization.
|
| |
|
| |
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
| |
PR: 113234
Submitted by: Andrej Zverev <andrey.zverev@electro-com.ru>
- Sync with actual script on pointyhat (whitespace, mostly)
|
|
|
|
|
| |
gracefully remove a node from the workload, and also it prevents build from
stucking when machine is deleted from mlist.
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
|
|
| |
just the plist ones. If the log is less than 1000 lines after the header,
include it all; else, trim to last 1000 lines.
This should help when deciding where to forward logs.
Tested on: pointyhat
|
|
|
|
|
|
|
|
| |
makes it possible to correctly analyze why packages were not built for a
specific run.
Add a beginning and ending email notification to help coordinate between
multiple portmgrs doing runs.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
lines has 3 spaces before SUBDIR word and all other categories has 4.
I've asked pav@ if there is a default format of category Makefiles and he said
the number of spaces doesn't matter, so, i fix addport to respect the current
number of spaces and/or tabs the file has.
Reported by: miwi, erwin
|
|
|
|
| |
Approved by: portmgr (self)
|
|
|
|
| |
Approved by: portmgr (self)
|
|
|
|
|
|
| |
* XORG_UPGRADE and USA_RESIDENT are no longer required
Approved by: portmgr (self)
|
|
|
|
|
| |
- Be more robust about param.h and allow space as well as tab as seperator
while determining OSVERION
|
| |
|
| |
|
|
|
|
|
|
| |
with portsmon reports.
Noticed by: erwin
|
|
|
|
| |
Discussed at: #bsdports
|
| |
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
|
| |
not errors:
discards qualifiers
[0-9]: implicit declaration of function
ISO C\+\+ forbids
|
|
|
|
|
|
|
| |
to defaults for PORTSDIR and PKG_DBDIR if the respective directory does
not exist, but bail out.
Approved by: netchild
|
|
|
|
| |
Glanced at by: kris
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- check if an installed libtool records dependencies recursively and
print a warning if it does
currently it prints the warning on every system which has libtool
installed from ports (only my local version doesn't do this, the
version in the ports is not correctly patched for this, a patch
similar in complexity (= simple) like the ltdl.m4 one in the
libtool-port-patch-directory is needed)
- enhance the regex which is responsible to not print a dependency to
the port we are just checking
- add a work in progress (not executed) to collapse the USE_* which
can have more than one value
neededlibs.sh:
- we also care about shared libs
resolveportsfromlibs.sh:
- take care about USE_OPENSSL, USE_EFL, USE_GL, USE_FAM, USE_OPENLDAP,
USE_SDL
- search in the "ldconfig -r" output if we can not find the lib ourself
- a better way of getting the first part of the LIB_DEPENDS stuff
(lib/libXYZ.so can be specified now too)
- some line wrapping + whitespace
- print the origin for the USE_* too (except USE_OPENSSL), so an user
can make some sanity checks and the explicit_lib_depends.sh can DTRT
if we check the USE_* port itself
- warn if we can not determine the right component (can happen for XORG)
unambiguously.
|
| |
|
| |
|
| |
|
| |
|
|
|
|
|
| |
- Print the portsdir variable in front of the origin for easy copy und paste
of the output.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
libs for a given installed port, e.g.:
# ./explicit_lib_depends.sh gnome-vfs-2.18.1_2
USE_GETTEXT=yes
USE_GNOME+=gconf2
USE_GNOME+=glib20
USE_GNOME+=gnomevfs2
USE_GNOME+=libxml2
USE_GNOME+=orbit2
USE_ICONV=yes
avahi-client:net/avahi
avahi-common:net/avahi
avahi-glib:net/avahi
dbus-1:devel/dbus
dbus-glib-1:devel/dbus-glib
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
dependencies of a port:
neededlibs.sh
Extract direct library dependencies (filenames) from binaries.
resolveportsfromlibs.sh
Prints the name(s) of ports(s) given a library filename,
suitable for direct use (copy&paste) in LIB_DEPENDS.
Example usage is included in the scripts. The following combined usage may
be helpful for further porting/testing automation:
resolveportsfromlibs.sh -b /usr/local $(neededlibs.sh /test/bin/*)
Requested by: kris, lofi (sort of)
|
|
|
|
|
|
|
|
|
| |
bsd.commands.mk and can be easily reused within the infrastructure.
- Revert old DESTDIR implementation.
- Add a new, fully chrooted DESTDIR implementation as bsd.destdir.mk.
Sponsored by: Google Summer of Code 2007
Approved by: portmgr (pav)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
zfs:
* Enabled by use_zfs=1 in portbuild.conf
* Populate build chroots by cloning a zfs snapshot instead of maintaining
many duplicate copies. In principle this is very efficient since
everything is copy-on-write and zfs snapshot creation is almost
instantaneous. There might be additional overheads from building on zfs
though. Currently the snapshot base is hard-wired to y/${branch}@base
but should be parametrized. This also must be populated beforehand, e.g.
during machine startup
* Clean build chroots by just destroying the snapshot.
tmpfs:
* Enabled by use_tmpfs=1 and tmpfs_size in portbuild.conf
* The previous md strategy of mounting in used/, populating and then
remounting (to avoid possible races from multiple builds claiming the
same chroot) doesn't work here because tmpfs instances are destroyed at
umount. I am not entirely sure the simpler approach is free from races.
|
|
|
|
| |
builds. Probably this should be better documented and commented ;)
|
|
|
|
|
|
|
|
|
|
|
|
| |
order to run certain host binaries that were kernel-dependent. We
now seem to be able to rely on the /rescue versions (and killall(1)
seems to be unused).
* Allow for ccache directories to be shared over NFS via the ccache_dir_nfs
portbuild.conf boolean
* Populate BSD.local.dist from ${PORTSDIR}/Templates and remove population
of BSD.x11-4.dist and support for XFree86 3.x
|
| |
|
|
|
|
| |
by its numeric equivalent since nc(1) does not look up the former.
|
|
|
|
|
| |
* Don't populate BSD.local.dist, this file is maintained in the ports tree
now and is populated at build time
|
| |
|
|
|
|
| |
* Add support for 6-exp2 experimental builds
|
|
|
|
|
|
|
|
|
|
| |
machine with the lowest number of running jobs. This worked when the
clients were all roughly equivalent, but schedules poorly when there
are some that are much more powerful (e.g. 8-core machines vs UP machines)
* We now compute the ratio of running jobs to maximum jobs and schedule on
the machine with lowest occupation fraction. This populates the machines
to equal fractions of their capacity.
|
|
|
|
|
| |
symlink ports and src trees elsewhere. With -l this would turn a populated
tree into a dangling symlink.
|
|
|
|
|
|
|
| |
* Only hardlink the old log files instead of anything else that might be
in the directories
* Add comment that old logfiles should be removed as well as packages, to
avoid duplicate versions of the same port log
|
| |
|
|
|
|
|
|
| |
run
* Add a 6-exp2 build which is a second experimental tree
* Remove an archaic 4.x/5.x log comparison
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
do it in portbuild from outside the jail thesedays
* Ignore /var/db/fontconfig which does not get restored to pristine state
* Save copies of master.passwd and groups and check them after the build
for changes, to look for user/group additions that may not be correctly
registered in UIDs/GIDs. Future work will hopefully automatically
check against those files and make unregistered IDs a fatal condition
* Correct logic mistake that was keeping distfiles for collection when
the checksum mismatched
|
| |
|
|
|
|
|
| |
Update default X11BASE. Retire PARALLEL_PACKAGE_BUILD and HAVE_MOTIF which
are now obsolete.
|
|
|
|
| |
have a more modern and complete version in the doc collection.
|
| |
|
|
|
|
|
| |
an 'f77' reason; flag 'warnings being treated as errors' as 'cc'. This
reduces the unknown output on i386-7 by about half.
|
|
|
|
|
|
| |
set it conditionally.
Submitted by: kris
|
| |
|
|
|
|
|
|
|
| |
variable TEMPLATES.
RT: 105108 (reworked from)
Submitted by: Curtis Jewell<swordsman@csjewell.fastmail.us>
|
|
|
|
|
| |
- Set X11BASE to ${LOCALBASE} for recent ${OSVERSION}.
- Bump PORTREVISION for ports intalling files in ${X11BASE}.
|
|
|
|
|
| |
Reported by: miwi
Tested by: miwi
|
|
|
|
|
|
|
|
|
|
|
| |
with very long arguments (>400000 characters).
The problem reveals for example if
/usr/ports/Tools/scripts/rmport -d print/ghostscript-gnu
is executed - it does
printf "%s\n" "... 451109 chars ..."
Spotted by: rafan
|
|
|
|
|
|
|
|
| |
packages due to packages being trimmed by RESTRICTED.
While here, note that the 'missing' column will be off by the number of
duplicates in the other columns. This happens when partial builds are
restarted.
|
| |
|
|
|
|
|
| |
in the repository. Suggested by: gabor@
* Fix typo in the output
|
|
|
|
|
|
| |
process. Thankyou openoffice!
Pass in the jail IP address in the JAIL_ADDR environment variable
|
| |
|
|
|
|
|
| |
Set DISPLAY to the IP address instead of using the :<...> implied form.
This is required for jails.
|
|
|
|
| |
uname
|
|
|
|
| |
the command line
|
| |
|
| |
|
| |
|
|
|
|
|
|
|
| |
on a machine that has use_md_swap=1, allow for the possibility of reusing
a md between builds if md_persistent=1. This requires a patch from pjd
to support BIO_DELETE in md devices, but it is a big optimization when
it can be used.
|
|
|
|
|
|
|
| |
will completely erase the list.
When using -continue, don't append to the duds.errors, we don't want to
preserve the old cruft from a previous build
|
|
|
|
| |
now regularly take longer than this :(
|
| |
|
|
|
|
| |
configure step.
|
|
|
|
|
|
|
|
|
|
| |
There is no change in any of the individual terms; this is merely a
rearrangement.
This change undoes what I was trying to do back in 2004 of breaking up
each individual test into a grep, for readability. The performance of
the script has continued to suffer as new greps were added over time,
to the point where this is now a bad tradeoff.
|
| |
|
| |
|
|
|
|
|
|
|
| |
directories, but a 5% loss on smaller ones.
No code changes (yet) except for the deletion of one duplicate
("fetch: transfer timed out" -> "fetch_timeout".
|
| |
|
|
|
|
| |
alignment apxs assert CATEGORIES cgi-bin forbidden stl
|
|
|
|
|
| |
we are seeing on the latest -current runs. Saves 40 seconds of time on
those runs, at the cost of 4 on the others.
|
|
|
|
| |
still have processes running, and add that into the highlight logic.
|
| |
|
| |
|
|
|
|
| |
readable and use double quotes everywhere.
|
| |
|
|
|
|
|
|
| |
and continue with removal anyway. Requested by miwi@
* Pipe dependencies information (if any) through a PAGER because INDEX lines
are very long and hard to read when wrapped
|
|
|
|
| |
latest gcc test build.
|
|
|
|
| |
web-interface becase the later is not reliable.
|
|
|
|
|
| |
If any are found then ask if the port should be skipped from removal or
removed regardless of the PRs.
|
|
|
|
| |
happens when security.jail.sysvipc_allowed=0
|
|
|
|
| |
except in disguise.
|
|
|
|
| |
timezone.
|
|
|
|
|
| |
these scripts on the cluster, transfer maintainership of these
to myself.
|
|
|
|
|
|
|
| |
This script can sometimes take several hours to run on builder,
and thus leading to confusing of why it still reports an error
that was fixed in cvs some time ago. Including the time when
the ports tree was updated should reduce some of this confusion.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
to be removed, possibly with the expiration date and deprecated reason
* If port is not marked for expiration than put "Removed" in ports/MOVED
entry instead of "Has expired"
* Implement -a option to remove all expired ports
* Ask if the cvs diff output should be recreated/reviewed again thus
giving the committer a chance to edit files by hand and view diff
results afterwards
* Cosmetic changes
|
|
|
|
|
| |
like .../barport/something e.g. expect either space, slash or eol after
port's name
|
|
|
|
|
| |
log' cell a link to it. My weak awk/sh skills are evident here, but it
does the job.
|
| |
|
|
|
|
| |
in addition to */*/Makefile* and ignore references in comments
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Add more XXX comments for future work
* Only record a cvsdone timestamp if we updated cvs
* When building with -trybroken, it's safe (and desirable) to run the
prunefailure script
* Reorganise a few things for better parallelism
* Instead of keeping a duplicate copy of the previous logs and errors
under bak/, just store a symlink to the archival location
* When doing an incremental build, also cycle out the old logs to avoid
broken links on the website (the logs from the previous build are
removed until the packages are rebuilt). Use cpio to create
hardlinked copies of the previous logs. XXX when these are bzipped
by cron to save space the links will be broken and it might actually
take more space.
* Don't bother bunzipping old logs, now that the processlogs scripts
can handle it. This was a waste of time anyway since they'd all be
rebzipped by the next nightly cron job.
* When the build is complete, stash a copy of the restricted ports in
bak/restricted/ before deleting them from packages/, and restore from
here when doing an incremental build to avoid needlessly rebuilding
them each time.
|
|
|
|
|
|
|
| |
127.0.0.0 or 127.0.0.1
* Don't use chroot to mount the linprocfs, or it won't be visible
within the jail.
|
|
|
|
|
| |
* More robust extraction of the header values (copes with the addition of
the build uname to the header)
|
|
|
|
|
|
|
| |
* Support .log.bz2 files
* Clarify that BROKEN ports are not frequently rebuilt so probably
won't appear
|
| |
|
|
|
|
|
|
|
|
|
|
| |
* Increase sparc64 build timeout to 24 hours (we have so few build
machines that we cannot afford to tie them up for longer)
* Increase other arch build timeout to 100 hours (hello openoffice!)
* If we successfully build a formerly broken package, touch errors/.force
which will kick off a rebuild of the html files
|
|
|
|
|
|
|
|
|
| |
* Use a generation number for the bindist tarballs, with compatibility
symlink. Eventually we'll use this to avoid building in a "stale"
chroot (i.e. populated by old world).
* Don't bother running ldconfig on i386, it is evidently not needed since
the other arches work fine without it
|
|
|
|
|
|
|
|
| |
* Don't try and mount/umount procfs, it won't work when we build inside a
jail.
* Report the uname -mr of the build environment, to ease confusion of
people reading the error logs by mail.
|
|
|
|
|
| |
* The 5.x variant for populating /etc does not work on 6.x and above, and
vice versa. Conditionalize old and new methods.
|
| |
|
| |
|
|
|
|
|
| |
dependencies. This needs some more polishing but for now it is better to
emit false positives than skip dependencies.
|
|
|
|
| |
fix true positive for "runaway_process".
|
|
|
|
|
| |
errors seen on -CURRENT as "compat6x" e.g. "someone needs to create a
compat6x port and then make it a dependency here."
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit should largele be a NOOP as it only adds support
for DESTDIR undefined. This does allow us to start testing
ports with DESTDIR set, but this is as of yet not supported.
Although this has been extensively tested on pointyhat, this
is a very intrusive change and some cases may have been
overlooked. Please contact Gabor and me if you find any.
PR: 100555
Submitted by: gabor
Sponsored by: Google Summer of Code 2006
|
| |
|
|
|
|
|
| |
Especialy i386 has enough horsepower to run two branches simultaniously.
This change has been on pointyhat for some time.
|
|
|
|
|
|
|
|
| |
${FILESDIR} which look like patches be treated as binary files. This
prevents RCS tags in patch fragments causing a problem for CVS.
Approved by: garga (maintainer),
ahze (mentor, implicit)
|
|
|
|
| |
a comment about openoffice.org*; generalize a bit.
|
| |
|
| |
|
| |
|
| |
|
|
|
|
|
| |
so do them separately in order not to break up the ease of browsing the
most important stats, above.
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
|
|
|
| |
break sed -E regular expressions
|
|
|
|
| |
to prevent spam
|
| |
|
| |
|
|
|
|
|
| |
number of packages built vs. packages that failed, along with some other
related information.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Remove obsolete explanations which are no longer seen, for speed:
ELF, MOTIF, MOTIFLIB, X_manpage, awk, bison, ffs_conflict, forbidden,
getopt, getopt.h, imake, lc_r, malloc.h, pod2man, sed, stl, soundcard.h,
texinfo, union_wait, values.h
- Add more cases to: arch, bad_c++, compiler_error, depend_object,
install_error, linker_error, mtree, perl5
These changes reduce many dozens of false positives; add a few dozen
true positives; and for certain directories, improve the speed about 10%
(a few drop by 15%).
It turns out that the performance issues are mainly due to the multiple
greps. If performance is an issue we need to go back to the moderately-
unreadable, everything-on-one-line paradigm. Before that happens, I would
like to experiment with some refactoring, so that the patterns are built up
in the shell line-by-line, so you could still be able to read it.
Tested on: pointyhat
Hat: portmgr
|
|
|
|
|
| |
Actually use ${SCP_DEST_TMP} to fix the case where
SCP_DEST_TMP != /tmp/
|
|
|
|
|
|
| |
When copying INDEX to the server, copy it first to a staging area and
first then to the real location. The copying can take long enough for
users to get a truncated file when downloading during the upload.
|
|
|
|
|
|
|
|
|
|
| |
time to add a new module. If you want to still use old way, just use
"-M freefall.FreeBSD.org" option
- Take addport maintainership
- When modulesupdate fail, ask user to retry
- Change modulesupdate to work fine with addport
Approved by: will (maintainer)
|
|
|
|
|
|
|
|
| |
filesystem (either absolute or relative) in addition to `category/port'.
PR: ports/96649
Submitted by: Cheng-Lung Sung <clsung@freebsd.org>
Reworked by: vd
|
|
|
|
|
|
| |
When removing category/port - look if other ports' Makefiles contain
`/port' rather than `category/port', since the later misses things
like `${.CURDIR}/../port'
|