Commit Graph

170 Commits

Author SHA1 Message Date
Pacman99
ba01aa7db7 mkSuites: generalize for one profile/suite pair 2021-04-26 11:27:00 -07:00
Pacman99
59383e871f modules: drop any logic already done by fup
this includes creating multiPkgs, and dropping options already set by
fup
2021-04-26 11:27:00 -07:00
Pacman99
abd133c244 make devosSystem construct a proper nixos builder
Get `self` and `inputs` during construction, and rely on
specialArgs.channel for nixos flake
2021-04-26 11:27:00 -07:00
Pacman99
2cab5b5d2b add and use lib.collectProfiles
this function collects profiles recursively
2021-04-25 20:53:16 -07:00
Pacman99
58c7d04036 extract iso/hm config modules to lib.modules 2021-04-24 14:44:32 -07:00
Pacman99
4e28ec2d8e devosSystem: fix iso build - can't remove core 2021-04-24 14:44:17 -07:00
Pacman99
2d9ea0d27e lib: init modules and move mkHosts modules there
This helps to split up the code in mkHosts and creates a place where we
can store modules relevent to devos. It will also be easier to remove
unecessary parts of each module in the future when they are all
compartmentalized.
2021-04-24 14:44:16 -07:00
Pacman99
a53aa8b7eb lib: drop jobs output, prefer checks and packages
With mkFlakeDoc in packages there is no need for jobs. And I think
anything that could go in jobs really should go in checks or packages.
If something needs to be tested - checks, if something needs to be built
- packages. jobs is not multi-arch and is redundant to build/test
things with when official flake outputs exist
2021-04-24 13:08:33 -07:00
Pacman99
1cd4ed136a mkFlakeDoc: move to packages output
the packages output allows us to do `nix build .#nixosOptionsDoc`.
theres also already an instantiated nixpkgs available there.
2021-04-24 13:08:06 -07:00
Pacman99
eab0bf074c lib: one line for arguments, drop userFlake*
Also format all files and add a flake.lock for lib
for a folder thats meant to work on other flakes theres never a reason
it should need to refer to itself, only other flakes. So "self" and
"inputs" are better namings for these variables. The userFlake* is
redundant and confusing, when trying to call the functions its hard to
figure out how to use them when there are now two lines of arguments to
figure out.
2021-04-23 23:46:56 -07:00
Pacman99
fe9ba26561 evalArgs: cleanup module and type references
fix pathTo and coercedList types
add modulesModule to also include modules option under home
2021-04-23 17:59:01 -07:00
David Arnold
eea4e40d7e
ref: config -> hosts | nixos -> os
in devos, we differentiate clearly between home and os
configuration, reason for which we are more precise by not naming
after the (more generic) fup API.
2021-04-21 21:45:39 -05:00
Pacman99
f3defb486d used coercedTo for typing and improve options
allow lists, nested lists, and non-lists for list like options
drop config.<name>.externalModules
2021-04-21 18:38:58 -07:00
Pacman99
24dbb2b323
add mkFlakeDoc to pkgs-lib to build options doc 2021-04-19 00:41:20 -05:00
David Arnold
239386b5f8
Merge branch 'core' into api-next 2021-04-19 00:10:47 -05:00
David Arnold
a714cf466d
fix: various left-overs 2021-04-18 23:48:19 -05:00
David Arnold
8134350545
ref: simplify pkgs-lib deps injection 2021-04-18 22:40:53 -05:00
David Arnold
362cc31827
fix: constructors of mkFlake function family 2021-04-18 22:29:28 -05:00
David Arnold
6f0392b55e ref: cave out instances of userFLake dependencies and intject them as
if functions where contructors
2021-04-18 22:16:53 -05:00
David Arnold
40acfd13e3 use: makeExtensible 2021-04-18 22:16:53 -05:00
David Arnold
be924bcb27 ref: reduce exposure to callLibs
for clarity's sake, expose which function uses final and prev, so that
people can have a clearer understanding how they relate to each other
in terms of dependencies.

also a simple `{ lib = final; }` probably does not warrant a complete
callLibs obscurization.
2021-04-18 22:16:53 -05:00
Timothy DeHerrera
f254304796
mv nix/ci.nix directly to default.nix 2021-04-18 21:00:45 -06:00
Timothy DeHerrera
127cb25bd1
lib: move compat to lib 2021-04-18 20:51:47 -06:00
Timothy DeHerrera
fbce126f4d
cachix: move out of root and into profile 2021-04-18 20:43:37 -06:00
Timothy DeHerrera
2d91946745
doc: move all docs under doc 2021-04-18 20:42:49 -06:00
Timothy DeHerrera
048171ecaf
lib: improve profileMap for added convenience
Simplify the definition of suites a bit by removing the need for list
concatenation.
2021-04-18 20:42:43 -06:00
David Arnold
9dca402914 ref: make onion with flake-utils 2021-04-18 20:31:49 -05:00
David Arnold
6cccb55263 ref: userSelf -> userFlakeSelf 2021-04-18 20:04:19 -05:00
David Arnold
cd7fb4f54c ref: flatten out inputs 2021-04-18 20:04:19 -05:00
David Arnold
16b3fad559 ref: merge dev into lib 2021-04-18 19:28:03 -05:00
David Arnold
c24199649a fixup: distinguish self and userSelf 2021-04-18 19:11:26 -05:00
David Arnold
6116779b23 fixup: nix flake check II / II 2021-04-18 19:11:26 -05:00
David Arnold
21a03fa94c fixup: nix flake check I / X 2021-04-18 19:11:26 -05:00
David Arnold
5f89d27428 ref: extract lib into subflake 2021-04-18 19:11:26 -05:00
David Arnold
863c17621c libtests: outfactor in preparation of lib/flake.nix 2021-04-18 19:11:26 -05:00
Pacman99
0db2bb041e add old evalArgs as evalOldArgs, so flake works 2021-04-13 11:24:10 -07:00
Pacman99
2b70cd3ae6 fix some small bugs in mkFlake/evalArgs 2021-04-11 22:34:15 -07:00
Pacman99
b7bcaea497 Update evalArgs to match the new planned api 2021-04-11 20:09:47 -07:00
Pacman99
658cd2bb08 pkgs-lib/shell: update to new homeConfigurations
flk should switch to use home-manager --flake whenever the updated
package is made available in the hm flake or nixpkgs
2021-04-11 10:28:28 -07:00
Pacman99
401566af34 lib/pkgs-lib: update checks for new hm format 2021-04-10 22:13:54 -07:00
Pacman99
8ba7bffc3f export users home for home-manager --flake
home-manager --flake needs the activationPackage in the root of the
homeConfiguration
2021-04-10 22:04:49 -07:00
Pacman99
ac4c26755b init pkgs-lib, move tests and shell there 2021-04-10 13:36:54 -07:00
Timothy DeHerrera
be4b612b23
treewide: format with nixpkgs-fmt 2021-04-09 19:22:08 -06:00
Pacman99
3daa990728
fix userSpecialArgs defaults (#234)
Co-authored-by: Pacman99 <pachum99@gmail.com>
2021-04-09 12:03:42 -05:00
Timothy DeHerrera
18dbc7d0e4
Revert "drop toString in evalArgs - trying to fix CI"
This reverts commit a253033931.
2021-04-08 21:18:08 -06:00
Timothy DeHerrera
eaff91c0f9
Revert "drop toString in pathsIn - trying to fix CI issues"
This reverts commit 930b3e96b4.
2021-04-08 21:17:24 -06:00
bors[bot]
3823eb6423
Merge #231
231: Move flake implementation logic to lib r=nrdxp a=Pacman99

This is a simpler version of #218 that moves flake logic to lib and adds a module to evaluate devos. This DOES NOT support out of tree usage, so if you were following any of the previous PR's, the doc sections/examples to use devos as a library will not work. There is work to make a cleaner api and only then will out of tree support work. Until then, this is still useful to simplify devos and clean up a lot of the implementation logic.

Co-authored-by: Pacman99 <pachum99@gmail.com>
2021-04-09 02:58:30 +00:00
Pacman99
115f1e6711 make core an unecessary profile and add to suites
Prevents mkHosts subverting standard devos api to import core and add
all necessary core features to mkHosts, so core can be safely deleted in
suites
2021-04-06 19:00:57 -07:00
Pacman99
a253033931 drop toString in evalArgs - trying to fix CI 2021-04-06 17:33:05 -07:00
Pacman99
930b3e96b4 drop toString in pathsIn - trying to fix CI issues 2021-04-06 17:02:30 -07:00
Pacman99
340784df6d lib: init evalFlakeArgs for mkFlake args
module describing devos flake arguments
2021-04-06 12:45:04 -07:00
Pacman99
a76ca177fd lib: init mkFlake to create a devos flake
general lib function - not devos specific
2021-04-06 12:45:04 -07:00
Pacman99
3d1501c384 lib: only readDir if path exists with safeReadDir 2021-04-01 11:39:02 -07:00
bors[bot]
f14dcdaf78
Merge #224
224: Allow for directories in module-list r=nrdxp a=Pacman99

fixes #221 
building on #222 this PR improves the lib functions pathsToImportedAttrs and pathsIn. First to add support for directories. This does not support actually passing a file in a directory, so `./matrix/default.nix` won't work but `./matrix` will - I should probably document this somewhere. 
Also I moved the filtering for nix files to `pathsIn`, since its only necessary for auto-import. We can assume that users would pass proper files in `module-list.nix`.

Co-authored-by: Pacman99 <pachum99@gmail.com>
2021-03-29 17:22:20 +00:00
Pacman99
7650526407 isoConfig: fix disabling profiles
convert each to a list which doesn't get appended to modulesPath
2021-03-29 09:59:58 -07:00
Pacman99
ec7eff63eb move hosts implementation logic to lib as mkHosts 2021-03-29 09:05:54 -07:00
Pacman99
7ebd6b5730 move suites implementation logic to lib as mkSuites 2021-03-29 09:05:54 -07:00
Pacman99
8ab865a853 lib/mkProfileAttrs: don't change to path
store paths can't be appended to a path
2021-03-29 09:05:54 -07:00
Pacman99
b3aa38702b lib/pathsToImportedAttrs: allow directories
check if directory has a default.nix and use directory name as key

Co-authored-by: Timothy DeHerrera <tim.deh@pm.me>
2021-03-29 08:54:25 -07:00
bors[bot]
ef1ee6e6b8
Merge #217
217: lib/mkPkgs: don't import external files and get inputs as argument r=nrdxp a=Pacman99

Changes mkPkgs to take extern, overrides, and pkgs as an argument. This improves their ability to be used as lib functions.

Co-authored-by: Pacman99 <pachum99@gmail.com>
2021-03-27 17:31:26 +00:00
Pacman99
562f476fb5 lib/mkPkgs: fix function arguments
take extern and overrides as arguments as to not import external files
get inputs as a function dependency from callLib
2021-03-27 10:09:05 -07:00
bors[bot]
5a3bae7be5
Merge #220
220: Drop flattenTreeSystem and use custom function for filtering packages r=nrdxp a=Pacman99

I don't think we should flatten the system because if a user doesn't make a package a derivation in pkgs/default.nix we should trust that there is a reason for doing so. So instead this drops the flattenTreeSystem reference(and switches to flake-utils master branch) and replaces its usage with a custom function `filterPackages`. This function filter all packages that match three conditions;
 - is a derivation
 - not broken
 - system is supported

In that order as to not cause errors when trying to reference non-derivation meta attributes.

And then also just dump *all* packages into legacy packages, so everything else is still accessible. I was considering removing the packages that are already in the packages output in legacyPackages, but I don't think its necessary since nix looks to the packages output first.

Co-authored-by: Pacman99 <pachum99@gmail.com>
2021-03-27 00:10:58 +00:00
Pacman99
fb14e20249 lib/filterPackages: filter by flake requirements
returns only non-broken derivations that support the system.
2021-03-26 16:55:27 -07:00
bors[bot]
3c772df5b0
Merge #215
215: lib/pathsToImportedAttrs: discard string context in the names to prevent flake check error r=nrdxp a=Pacman99

So I learned about this myself while working on it. Nix has a thing called string context, where strings refer to various store paths that they used to contain. This is normally a good thing to guarantee purity, but flake outputs cannot contain store paths. So when `pathsToImportedAttrs` is passed a list of store paths, and is then used for a flake output, we get a very confusing error that the string refers to a store path - even though it doesn't look like it does! 
I think its good to review this as a separate Pr, since it requires calling an unsafe nix builtin.

Co-authored-by: Pacman99 <pachum99@gmail.com>
2021-03-26 21:30:04 +00:00
bors[bot]
00759997ab
Merge #211
211: home-manager fixes r=nrdxp a=nrdxp

- [x] integrate nix-community/home-manager#1880 (to avoid duplicate commits, we'll wait a bit to see if this gets merged)
- [x] don't use all of `systemPackages` for `flk home` 
- [x] test home-manager `activationPackage` and `flk home` as part of CI

Co-authored-by: Timothy DeHerrera <tim.deh@pm.me>
2021-03-26 21:22:49 +00:00
bors[bot]
205a9d3e03
Merge #219
219: lib/mkPkgs: reorder overlays to get access to devos lib(dev) r=nrdxp a=Pacman99

nixpkgs applies overlays in the order given. This PR moves the lib extension overlay to be first so all subsequent overlays can access `dev` which is the devos lib.

Co-authored-by: Pacman99 <pachum99@gmail.com>
2021-03-26 21:07:51 +00:00
Pacman99
d344fc2ea6 lib/mkPkgs: reorder overlays to get access to devos lib(dev) 2021-03-26 13:00:38 -07:00
Pacman99
68a5a2eecf lib/pathsToImportedAttrs: discard string context in the names to prevent flake check error 2021-03-26 08:10:56 -07:00
Pacman99
61fc640f10 lib/pathsIn: don't error if path doesn't exist 2021-03-25 15:21:10 -07:00
Timothy DeHerrera
e767f7066e
devosSystem#hmConfig: don't pass systemPackages
The intentions are good, but for users with many profiles, this is
overkill. Proper solution will come when #127 is resolved.
2021-03-25 00:17:50 -06:00
Pacman99
dc9d9877b8 export homeConfigurations for all user@host pairs and add better home-manager only defaults 2021-03-23 09:43:37 -07:00
bors[bot]
870897a3e9
Merge #202
202: iso: avoid systemd service startup r=nrdxp a=blaggacao

fixes #194
alternative to #197

# Manual Tests


<details>
<summary>was unrelated</summary>

- [ ] `flk install NixOS --impure` correctly onto `/mnt`  (looks like no profile is present)

Issue: https://github.com/divnix/devos/issues/204
Upstream Issue: https://github.com/NixOS/nixpkgs/issues/116938

</details>

- [x] acceptable build time / closure size ca 850MB (for a simple base OS) ✔️ 
- [x] local profile with `cage` service is disabled, that is: boots into terminal ✔️ 
- [x] success: air gapped / offline devshell enter ✔️ 
- [ ] failure: aire gapped target install: &larr; non blocking bonus item  
```console
$ flk install POS
warning: you don't have internet access; disabling some network-dependent features
building the flake in path:/iso/devos?narHash=sha265-...
warning: you don't have internet access; disabling some network-dependent features
error: unable to download 'https://api.github.com/repos/NixOS/nixpkgs/df8e3...': Couldn't resolve host name (6)
```

&rarr; detailed rationale in the commit messages

❤️ @Pacman99 for the excellent and detailed discussions in #197 and the may ideas, suggestions and code.

Co-authored-by: David Arnold <dar@xoe.solutions>
2021-03-22 21:59:12 +00:00
Pacman99
aec7c92cc7 hosts: set nixpkgs.pkgs based on nixpkgs.system 2021-03-19 12:23:23 -07:00
David Arnold
49b1ad8227
iso: ensure tools of deactivated profiles are still available
This is just for convenience, since the closuers are already in the
store. It might be helpful to be able to test out some things
of those deactivated profiles een on the iso isntaller.
2021-03-18 23:46:47 -05:00
David Arnold
007447652b
iso: add back complete host profile to the nix store
This ensures that all builds of activated profiles are included into
the iso cache and don't require rebuilding within the live installer
environment.
2021-03-18 23:46:47 -05:00
David Arnold
435847823f
style / organization 2021-03-18 23:46:47 -05:00
David Arnold
15cf15b3ed
iso: filter out al profiles (except core)
IN order to avoid random startup of systemd services, filter out all
profiles, except for core and user profiles.

This works becasue of a fundamental devos contract, that modules
only define configuration, but don't implement them and profiles
only implement confguration but don't define them. So only ever an
activated profile is expected to effectively start up a systemd service.

closes: #194
2021-03-18 23:46:47 -05:00
David Arnold
db716d1921
lib: ensure path type in mkProfileAttrs
This is required so that filtering via lib.remove works against 
modules.core and similar which are of path type.

It is also a prerequisite for disabledModules to match by module.key
instead of path string relative to nixpkgs' modulePath.
2021-03-18 23:46:47 -05:00
Pacman99
dd494facc3 hosts/devosSystem: pass modules as attrset 2021-03-17 17:28:49 -07:00
David Arnold
8e3777ff34
iso: copy devShell storepath to iso
This avoids for the live nixos on the iso to rebuild the devshell
upon entering it.
2021-03-16 17:35:58 -05:00
David Arnold
17aee57df4
iso: pin inputs as flakes
This intends to avoid downloading inputs on the iso live image
and is a step towards completely air-gaped installs.

credits: @Pacman99

closes: #190
2021-03-16 17:12:38 -05:00
David Arnold
3d324e7533 lib/devos: bake devos repo into live cd (#168)
fix #167 

This worked for me to bootstrap another machine.
2021-03-16 19:50:38 +00:00
Timothy DeHerrera
d8068073c6
lib: add rgxToString function 2021-03-14 21:40:49 -06:00
Timothy DeHerrera
d51cd34fb7
subflakes: make first class citizens
Subflakes should provide their wares as outputs, so wire up the pkgs
flake to reflect that.

Due to the unstable nature of flakes, updating the root flake doesn't
currently update the subflake lock file. Therefore, add additional
logic to flk update script in order to do this behind the scenes.

Nix is now pulled in from the "nix" registry flake in order for users
to take advantage of improvements to the UI since its last update in
nixpkgs.
2021-03-14 21:27:58 -06:00
Timothy DeHerrera
c012f2f4ed treewide cleanups and refactoring for initial tests (#157)
- [x] refactor lib into separate files, similar to NixOS/nixpkgs/lib.
- [x] refactor ci to automatically generate derivations from flake outputs
- [x] remove cluttered indirection statements throughout the codebase
- [x] refactor hosts to allow for upcoming integration tests
- [x] improve ambiguity in the existing docs 
- [x] add [BORS](https://bors.tech) support
- [x] add initial integration test
- [x] write tests documentation
- [x] test lib
- [x] improve version string generation, and do so automatically for pkgs/flake.nix sources

Clean up the codebase as best we can in preparation for #152 and add tests. From now on, all PRs will be merged with BORS.
2021-03-14 07:10:51 +00:00
Timothy DeHerrera
5bd8876b74
Merge pull request #147 from blaggacao/da/lib-with-dep-on-pkgs
lib: can depend on pkgs (a la nixpkgs#pkgs/pkgs-lib)
2021-03-05 12:16:38 -07:00
Timothy DeHerrera
7c226c9168
lib#genPkgs: reorder overlays
Resolves #146 by allowing external overlays to ve overriden by local
overlays.
2021-03-01 22:15:48 -07:00
David Arnold
e56c619aef
lib: add usage docs 2021-03-01 14:37:43 -05:00
David Arnold
bca532ab42
lib: can depend on pkgs (a la nixpkgs#pkgs/pkgs-lib) 2021-03-01 14:01:17 -05:00
Timothy DeHerrera
25180a5e6e
pkgs: use subflake to manage package sources
* Resolves #118
* Leverage flakes to manage package sources & hashes
* Update documentation with an example.
* Add `mkVersion` function to autogenerate a version string.
* Add srcs package via overlay containing all sources defined in
  _pkgs/flake.nix_
* Extend `flk update` with the ability to only update the given input
2021-02-26 00:15:45 -07:00
Timothy DeHerrera
1651913910
profiles: simplify profiles to suites
* Leave importing to nixpkgs module implentation. Provide a path
  instead; resolves #136.
* Allow profiles which are not lambdas but simple attribute sets,
  relaxing the constraints a bit.
* Update profile README.md
* defaultImports -> mkProfileAttrs: allow importing subprofiles even
  if parent directory does not contain a default.nix.
2021-02-25 14:47:19 -07:00
Timothy DeHerrera
77296ff433
deploy: move logic to lib 2021-02-15 11:24:43 -07:00
Timothy DeHerrera
cf23b8e64d
flake: rename flake-utils input 2021-02-14 22:11:49 -07:00
Timothy DeHerrera
bf34d2071f
doc: new mdbook documentation 2021-02-14 02:46:05 -07:00
Timothy DeHerrera
3bc16596f7
lib: rename defaultImports 2021-02-06 13:30:17 -07:00
Timothy DeHerrera
0ec0c25238
various refactors
* Move extern lists to their own folder
* Move unstable package and module imports to their own folder
* Create a genPkgs function to avoid using legacyPackages for the whole
  package set
* Move hmActivationPackages to legacyPackages
2021-02-03 23:44:58 -07:00
Timothy DeHerrera
39acbefbe9
ci: fix ci after core creation 2021-02-03 19:35:10 -07:00
Timothy DeHerrera
bd9878f82a
ci: move into build attribute 2021-02-03 16:41:21 -07:00
Timothy DeHerrera
c67b846929
profiles: add concept of suites
* Import attrs of profiles automatically with `defaultImport`.
* Refactor profiles to ensure all are functions returning a module.
* Add a suites.nix with collections of profiles.
* Add suites as `specialArgs` to modules.
* Add suite import to NixOS host.
2021-02-03 13:16:14 -07:00
Timothy DeHerrera
4ae99af29d
Merge pull request #102 from blaggacao/da/iso-sd-netboot
host: add deault implementation for system.build.isoImage target (per…
2021-02-01 19:51:44 -07:00