I originally built on ubuntu 16.04 LTS just hacking my way through without really documenting my changes. That system has since been upgraded to ubuntu 18.04 LTS. @egc has been working on building for northstar on a recent ubuntu distribution with some assistance from me but didn't finish yet. It's a long frustrating process. If a few people are interested in joining @egc we could all cooperate.
Thanks for the info @kernel-panic69, didn't know the strings trick no longer worked for identifying a build's toolchain.
FWIW, I followed a pretty similar path, starting with Ubuntu 16.04 LTS and now on 18.04 LTS, but sadly I too kept scant documentation (including why the upgrade to Ubuntu 18 in the middle).
A common GIT repo would be a great help, but the DD-WRT build is its' own enemy scribbling over the source tree on a regular basis, so getting decently reproducible builds is harder than it needs to be. But it would be great to at least be able to get it building, with hook or crook.
Sounds like a lot of work to keep a git repo up to date with BS's svn which changes rapidly. BS is very productive.
Couldn't we just create/share a few scripts and svn patches that setup the build environment.
1) Start with one host eg. ubuntu 18.04 LTS
2) Start with one target eg. Northstar
3) Give the community build a name K2: Savage Mountain (not Kong 2 )
4) Share scripts/patches/toolchain(s)
5) Try to get BS to submit useful features/fixes we come up withBS is not unreasonable when presented with changes/fixes. Just from my suggestions he fixed cron, missing lines in syslog viewer, an error when stopping privoxy and even a Makefile only fix in iptables.
Having an easy way for the community to build will attract/keep highly skilled users improving dd-wrt benefiting all users.
Sounds like a lot of work to keep a git repo up to date with BS's svn which changes rapidly. BS is very productive.
Couldn't we just create/share a few scripts and svn patches that setup the build environment.
1) Start with one host eg. ubuntu 18.04 LTS
2) Start with one target eg. Northstar
3) Give the community build a name K2: Savage Mountain (not Kong 2 )
4) Share scripts/patches/toolchain(s)
5) Try to get BS to submit useful features/fixes we come up withBS is not unreasonable when presented with changes/fixes. Just from my suggestions he fixed cron, missing lines in syslog viewer, an error when stopping privoxy and even a Makefile only fix in iptables.
Having an easy way for the community to build will attract/keep highly skilled users improving dd-wrt benefiting all users.
I think that's a good plan overall (I'm actually starting on a similar list with a clean Ubuntu 18 VM and brand new checkout of DD-WRT), I just know how quickly the "shared scripts and patches" leads to a mess where you can't figure out which the right scripts / order / ... is. Maybe a halfway point would be a Git repo or at least Wiki for the scripts/patches?
Joined: 08 May 2018 Posts: 7446 Location: Texas, USA
Posted: Sat Feb 29, 2020 17:02 Post subject:
I would say limit it to Broadcom-only in general with only the kernels required - 2.4, 2.6, 3.10, 4.4(?), maybe 3.18 to keep the overall repo size down. I think quarkysg called it 'selective checkout' with SVN, limiting it to only the code required. Most public resources (github, bitbucket) limit the size of a repository to a certain size unless you pay $$$, if I am not mistaken. It doesn't have to be 'public', you can also limit access to specific users, if I am not mistaken. I would have to set up another separate dev environment altogether to do anything, anyway. Honestly, it doesn't matter what revision control system you use, maintaining it with as much activity goes on with DD development will be a full-time job. I'm open to whatever. Speaking of quarkysg, I ain't seen him around in a minute, I think we lost him to the temptation of OpenWRT?
rkboni wrote:
I think that's a good plan overall (I'm actually starting on a similar list with a clean Ubuntu 18 VM and brand new checkout of DD-WRT), I just know how quickly the "shared scripts and patches" leads to a mess where you can't figure out which the right scripts / order / ... is. Maybe a halfway point would be a Git repo or at least Wiki for the scripts/patches?
Maybe a halfway point would be a Git repo or at least Wiki for the scripts/patches?
Sounds good. That way svn updates to BS's latest would be mostly effortless and if a patch conflicts, one person fixes the patch and all benefit. Kong must have had something similar set up but unfortunately never shared it.
github, gitlab, BitBucket, others... whoever takes the lead can choose.
Note that ubuntu 20.04 LTS is coming soon and will be supported for 5 years.
Attached a tweaked selective_co.sh that downloads one selected kernel only.
UPDATED: got a little bit further, now it looks like I'm stuck due to a bad build of my OpenWRT toolchain... going to re-spin that, but not tonight. In the meantime adding patches for the initial setup and the Makefile bug I found as attachments.
hanskloss wrote:
Here's my experience compiling dd-wrt from sources for the ARM platform. Previous posts were very helpful in setting up the build platform. My post will focus more on steps and errors I have encountered rather than providing "do-it-all" scripts.
In the example below I have compiled a mini image for R7000 SVN revision: 33941 Full image is not recommended at first as too many changes are required, to make it work.
After taking the notes, I have re-installed the OS and pulled fresh dd-wrt sources to make sure these steps are reproducible.
Since there's been conversation about getting builds running again, I thought I'd start here with a clean Ubuntu 18.04 LTS VM and see how far I get...
hanskloss wrote:
PREPARATION
What toolchain?
I downloaded BS and Kong's firmwares with the same kernel and grep'ed files to see what they used.
Code:
strings /lib/libstdc++.so.6 | grep toolchain
BS result: toolchain-arm_cortex-a9_gcc-6.3.0_musl-1.1.16_eabi
Kong result: toolchain-arm_cortex-a9_gcc-6.3.0_musl_eabi
This is clearly out of date for latest builds. Since I already had a GCC 9.2.0 toolchain build from OpenWRT's repo, I started off with that (toolchain-arm_cortex-a9_gcc-9.2.0_musl_eabi). This is probably also (slightly) out of date, as I built this at the end of Sept. 2019 to go with build r41027 (see https://forum.dd-wrt.com/phpBB2/viewtopic.php?p=1177650#1177650).
hanskloss wrote:
Linux distribution used: Debian Jessie amd64 8.9.0 DVD-1
Additional developer packages are needed to compile the mini version:
Minor updates here:
* strace.mk now exists, so left alone (line 217)
* oled.mk still missing (line 229)
* ath9k.mk moved to mac80211/ath9k.mk and exists, so left alone (line 240)
* atm.mk still missing (line 244)
hanskloss wrote:
Comment out Ralink drivers not used by our router, lines 287-296 in the Kconfig file
Code:
cd ~/DD-WRT/src/linux/universal/linux-4.4/drivers/net/wireless
Proprietary Broadcom wireless modules
Make following corrections because of non existent source file to compile wl.ko module
cd ~/DD-WRT/src/linux/universal/linux-4.4/brcm/arm/wl/config
Comment out in the file wl.mk line 448 WLFILES_SRC_HI += src/wl/clm/src/wlc_clm_data.c
Note that none of the wlc_<stuff>.c files seem to exist (but pre-build objects do). Maybe the right thing to do is not to build anything in this directory? Still, commented out as per instructions.
hanskloss wrote:
cd ~/DD-WRT/src/router
Comment out lines 181 182 inside Makefile.northstar where it copies wlc_clm_data.c
Just to be safe, commented out all instances of the cp/touch pattern (lines 170-171, 173-174, 179-180, 182-183).
hanskloss wrote:
cd ~/DD-WRT/src/linux/universal/linux-4.4/drivers/net/wl/
Append following to the line 95 $(SRCBASE_OFFSET)/wl/clm/src/wlc_clm_data.o in the file wl_generic.mk
Edit emf.mk Warning! make files are space/TAB sensitive. Need TAB before each install -D command.
cd ~/DD-WRT/src/router/rules
I needed to use serial recovery in my builds because included wireless modules would cause my router to "kernel panic" during the boot.
Replace emf.ko and igs.ko in the directory ~/DD-WRT/src/router/emf_bin/arm with modules which come from BS or Kong's firmware.
Good modules have the following sizes:
21864 emf.ko
16900 igs.ko
WebGUI
Remove optimizefiles from kromo/dd-wrt/Makefile line 1139
and is now line 1168. Removed optimizefiles as per instructions.
hanskloss wrote:
Final web interface will not be fully functional. Some solutions exists: using tsynik patches or extracting javascripts from the www binary. All asp and javascripts page sources are compiled into single www binary
~/DD-WRT/tools/filelist contains the exact listing of all files(web pages and images)
CONFIGURE
These packages need to be compiled first in this order. Kernel step will generate kernel.release file which is later needed for compilation of other packages.
Code:
make -f Makefile.northstar jansson-configure
make -f Makefile.northstar jansson
make -f Makefile.northstar nvram
make -f Makefile.northstar utils
make -f Makefile.northstar install_headers
make -f Makefile.northstar iptables
make -f Makefile.northstar libnet
make -f Makefile.northstar kernel
Fails on the last step:
Code:
make[1]: Leaving directory '/home/rafal/dev/DD-WRT/src/linux/universal/linux-4.4'
Makefile.northstar:166: recipe for target 'kernel' failed
due to:
Code:
CC arch/arm/kernel/bios32.o
In file included from arch/arm/plat-brcm/mpcore_core.c:25:
./brcm/arm/include/bcmutils.h:87:10: fatal error: hnd_pktq.h: No such file or directory
87 | #include <hnd_pktq.h>
| ^~~~~~~~~~~~
compilation terminated.
scripts/Makefile.build:277: recipe for target 'arch/arm/plat-brcm/mpcore_core.o' failed
I'm not sure if this is relevant at this point, since it does produce the kernel.release file and so I continued on...
hanskloss wrote:
Fix the typo in the libubox.mk line 7. Include dir should be json-c not json
cd ~/DD-WRT/src/router/rules
UBOX_EXTRA_CFLAGS=-I$(TOP)/_staging/usr/include -I$(TOP)/_staging/usr/include/json-c $(COPTS) $(MIPS16_OPT)
In the previous revisions compiliation of the libubox package frequently failed. Run these steps too ensure it compiles fine.
Code:
make -f Makefile.northstar json-c-configure
make -f Makefile.northstar json-c
make -f Makefile.northstar libubox-configure
make -f Makefile.northstar libubox
Final configure and all
Code:
make -f Makefile.northstar configure
make -f Makefile.northstar clean all
Next, I tripped over an error compiling in src/router/rc like so:
Code:
make[1]: Entering directory '/home/rafal/dev/DD-WRT/src/router/rc'
[CC] rc.o
arm-linux-uclibc-gcc: warning: environment variable 'STAGING_DIR' not defined
In file included from /home/rafal/dev/DD-WRT/src/router/shared/netconf.h:24,
from rc.c:26:
/home/rafal/dev/DD-WRT/src/linux/universal/linux-4.4/brcm/arm/common/include/proto/ethernet.h:29:10: fatal error: packed_section_start.h: No such file or directory
29 | #include <packed_section_start.h>
| ^~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
Makefile:1074: recipe for target 'rc.o' failed
make[1]: *** [rc.o] Error 1
make[1]: Leaving directory '/home/rafal/dev/DD-WRT/src/router/rc'
rules/all.mk:339: recipe for target 'rc' failed
This looks like it might be a bug in the rc/Makefile, which this change should fix:
My next failure isn't very much further and appears to be related to how I built my OpenWRT toolchain. I'm calling it a night but have an idea of what needs fixing, so will follow up soon.
hanskloss wrote:
INSTALL
Compile trx in opt/tools and copy to src/router/tools
These files are for Netgear and D-link. Asus and TP-Link need some fixes so I skipped those.
Original mksquashfs4 gives me illegal instruction error. I used the debian version which works fine.
Replace the line 575 in the Makefile.northstar
Code:
/usr/bin/mksquashfs $(ARCH)-uclibc/target $(ARCH)-uclibc/root.fs -comp xz -nopad -root-owned -noappend -Xbcj arm
Since trx tools were not compiled for TP-Link and Asus comment out the following lines in the Makefile.northstar
583-588 for TP-Link
655-664 for Asus
668-669 for Buffalo
Final command
make -f Makefile.northstar install
Look for the firmware inside this folder:
~/DD-WRT/src/router/arm-uclibc
Some tips
R7000 Recovery:
During the boot ctrl+ C
At the CFE>tftpd
tftp -m binary 192.168.1.1 -c put K3_R7000.chk
Example of compiling single kernel module
GCCARM=/home/user/toolchain-arm_cortex-a9_gcc-6.3.0_musl-1.1.16_eabi/bin
export PATH=$GCCARM:$PATH
cd ~/DD-WRT/src/linux/universal/linux-4.4
make ARCH=arm modules SUBDIRS=drivers/net/switch
Displaying variables or paths during compilation:
Insert into Makefile.northstar following line as needed.
$(info ******************)
$(info My ARCH is $(ARCH))
$(info ******************)
Joined: 08 May 2018 Posts: 7446 Location: Texas, USA
Posted: Sat Feb 29, 2020 20:53 Post subject:
@yoyoma2, it should be limited to only what is required for Broadcom platforms only, which may take some investigation, but that is a good start. You, egc, and rkboni probably need to compare notes and such.
@rkboni, if you haven't been following svn commits, you may run into more issues than what hanksloss annotated. There have been plenty of changes to Broadcom since his write-up.
Also, you should likely keep from choosing a specific revision to checkout, save and except for specific circumstances which will require back-tracking. It would be great if we had a different common place to discuss, but it is what it is at this point. I may look into setting up a github. What would be ideal is to clean out all of the unused cruft from the source tree so we are only looking at what is absolutely necessary, but that is semi-optional. The ideal goal would be to be able to submit patches to fix all Broadcom builds, not just Northstar. Anyhow, some more of my $0.02 on this. _________________ • Official Forum Rules, Guidelines, and Helpful Information • Firmware FAQ • Installation Wiki • Where Do I Download Firmware? •
DON'T use Chromium-based browsers • RTFM/STFW - TL;DR is NOT an excuse. • Why Should I Care What Color the Bikeshed Is? Please DO NOT PM me with questions; Ask in the forum.
---------------------------------------------------------
Linux User #377467 counter.li.org / linuxcounter.net
@yoyoma2, it should be limited to only what is required for Broadcom platforms only, which may take some investigation, but that is a good start. You, egc, and rkboni probably need to compare notes and such.
Given the rather opaque build structure and amount of code in the tree, I suspect that getting to "what is required for Broadcom platforms only" is a huge goal. But it would be nice not to have to drag all that code around!
kernel-panic69 wrote:
@rkboni, if you haven't been following svn commits, you may run into more issues than what hanksloss annotated. There have been plenty of changes to Broadcom since his write-up.
Also, you should likely keep from choosing a specific revision to checkout, save and except for specific circumstances which will require back-tracking. It would be great if we had a different common place to discuss, but it is what it is at this point. I may look into setting up a github. What would be ideal is to clean out all of the unused cruft from the source tree so we are only looking at what is absolutely necessary, but that is semi-optional. The ideal goal would be to be able to submit patches to fix all Broadcom builds, not just Northstar. Anyhow, some more of my $0.02 on this.
Oh, I'm sure there are more issues than annotated by hanksloss... But the SVN commits haven't been particularly instructive in terms of what's changing and particularly how it affects the builds. Part of my starting there was because it was a known point where things did actually build, and the other part was because others seemed to be starting there as well.
Re: checking out a specific version of SVN, I agree in principle... but given that we're way behind the 8-ball in terms of getting anything close to a sane build, I figured starting at a revision that's been known not to have build issues (because someone built it and people are using that build) would at least save some potential aggravation with trying to track a moving target. Once we get some revision building, then we can work on expanding the scope.
At least we now have a group of interested folks talking about it -- that's a great start. Any contributions, be they better ways to coordinate or communicate about this, pointers to other useful scripts/info, etc. are always welcome!
I wonder how important sticking to a compiling chain is (considering newer GCCs) to the consideration whether a workable image could be obtained. It may be due to some specific flags passed when building the cross-compiler to ensure low-level logics not optimised away probably. But my point is, I think for a better source tree, we probably have to think of how to eliminate all these non-generic elements in the repository. And that the repository should just reside a auto build script that checkouts GCC source tree and compile the toolchain in user environment with a `make ARCH=arm LIBC=musl cc`.
Leveraging Subtrees or Other Mechanism
My gut feeling would be we have to eliminate the Linux source trees and other packages source trees if we are to establish a new repository. The new repository would probably just include references to the relevant repositories and at the same time have reference to specific checkout that are known to be stable from upstreams e.g., the Linux source tree.
Other than these references, the repository then keeps the relevant patches or extra low-level header files, source files created specifically for some devices that are supposed to overlay to the upstream source trees. And once these patches are applied up-stream, they should be removed in the newer commits. We may need handle special cases where,
1) user requires earlier package version; DD-WRT patches already submitted upstream and eliminated from the current source tree.
2) user requires experimental (unstable) version that includes DD-WRT patches submitted upstream; DD-WRT stable version still contains the patches as the referring stable package version has not been including the patches.
We probably should diff-ing all the "external" source trees and find out how Kong and BS worked on the "external" source trees and establish relevant policies to how our tree should overlay to the "external" source trees.
My expectation would be either using menuconfig for users to select whether building DD-WRT taking defaults (all stable packages and kernel), e.g.,
Code:
make config
Are you to build the DD-WRT firmware with known stable kernel and packages? [Y/n]: n
Are you to build the DD-WRT firmware with recognised stable Linux 4.4.208? [Y/n]: y
Are you to build the DD-WRT firmware with recognised stable package bird [commit xxxxxxx] [Y/n]: n
Please specify the commit to check out for building bird: yyyyyyy
...
Checking
We probably need more autoconfig scripts to ensure pre-requisites are met for compiling DD-WRT source tree.
Separation of Generic Code and Device Specific Configs
We also need to establish some mechanism to ensure reuse of generic attributes. Probably tuple-based config files that says a specific device should supports some features and should compile a certain package; or that a device should apply certain compilation FLAGS etc. By then, if a user issue `make EA6900` would build a stable version of binaries from DD-WRT source tree, that also checkouts all recognised stable versions of packages and Linux builds.
E.g.,
device.arch.conf
ea6900 : armv7 vfp3
e2000 : ...
I am interested in working alongside many of you if we are to establish a new source tree.
I see! Which means it's more like fixing the current SVN repositories than to have a revamp of the code base.
What you're describing is very much the OpenWrt model where most of the code lives external to the repo and and just package lists and generic build scripts handle most of it. I'm pretty sure they still have their own repos for some things but generally their builds are driven off of external sources, local patches and local build config.
DD-WRT trades that for an all inclusive source tree that also includes vendor firmware / binary driver blobs and thus supports lots more hardware.
Different models for different folks.. while I admire the simplicity of OpenWrt it's never supported much of my router hardware, which is why I've been running DD-WRT forever. But whichever you run, you're unlikely to be able to turn one project into the other.
What you're describing is very much the OpenWrt model where most of the code lives external to the repo and and just package lists and generic build scripts handle most of it. ...
DD-WRT trades that for an all inclusive source tree that also includes vendor firmware / binary driver blobs and thus supports lots more hardware.
... while I admire the simplicity of OpenWrt it's never supported much of my router hardware, which is why I've been running DD-WRT forever. ...
I believe it's rather irrelevant to whether the reliance on external source trees "OpenWRT" model traded support but rather it is OpenWRT's philosophy to insist their compiled binaries must be based on Open Source Softwares that hinders its support on from their compiled binaries.
Lack of device support is also the reason why I started to use DD-WRT, especially with Broadcom devices that cannot go beyond 802.11g . But purely out of academic discussions, I believe it would be do-able DD-WRT taking the "OpenWRT" source control model while continuously support more and more devices as long as this project is not following OpenWRT rigidity in compiling only OSS, i.e., supporting proprietary blobs in the source tree.
But that said, I wonder if having a side-by-side "moderated" repository that occasionally synchronise codes with external sources would benefits the wider community by generating personalised firmwares based on "generic" source tree. This "moderated" repository and the "developer" DD-WRT would be something analogous to RedHat and Fedora, or CentOS and Fedora relationship.
And by the way, I believe I read something earlier that BrainSlayer was trying to merge/sync the code with OpenWRT but I wonder what is the scope or how it goes?
)
