Compiling DD-WRT

Post new topic   Reply to topic    DD-WRT Forum Forum Index -> Generic Questions
Goto page Previous  1, 2, 3 ... 15, 16, 17, 18  Next
Author Message
rkboni
DD-WRT Novice


Joined: 04 Mar 2016
Posts: 49

PostPosted: Sat Feb 29, 2020 4:16    Post subject: Reply with quote
yoyoma2 wrote:
I originally built on ubuntu 16.04 LTS just hacking my way through without really documenting my changes. That system has since been upgraded to ubuntu 18.04 LTS. @egc has been working on building for northstar on a recent ubuntu distribution with some assistance from me but didn't finish yet. It's a long frustrating process. If a few people are interested in joining @egc we could all cooperate.

Thanks for the info @kernel-panic69, didn't know the strings trick no longer worked for identifying a build's toolchain.


FWIW, I followed a pretty similar path, starting with Ubuntu 16.04 LTS and now on 18.04 LTS, but sadly I too kept scant documentation (including why the upgrade to Ubuntu 18 in the middle).

A common GIT repo would be a great help, but the DD-WRT build is its' own enemy scribbling over the source tree on a regular basis, so getting decently reproducible builds is harder than it needs to be. But it would be great to at least be able to get it building, with hook or crook.

Sorry for the rant!
Sponsor
yoyoma2
DD-WRT User


Joined: 24 Sep 2016
Posts: 146

PostPosted: Sat Feb 29, 2020 15:37    Post subject: Reply with quote
Sounds like a lot of work to keep a git repo up to date with BS's svn which changes rapidly. BS is very productive.

Couldn't we just create/share a few scripts and svn patches that setup the build environment.
    1) Start with one host eg. ubuntu 18.04 LTS
    2) Start with one target eg. Northstar
    3) Give the community build a name K2: Savage Mountain (not Kong 2 Wink)
    4) Share scripts/patches/toolchain(s)
    5) Try to get BS to submit useful features/fixes we come up with
BS is not unreasonable when presented with changes/fixes. Just from my suggestions he fixed cron, missing lines in syslog viewer, an error when stopping privoxy and even a Makefile only fix in iptables.

Having an easy way for the community to build will attract/keep highly skilled users improving dd-wrt benefiting all users.
rkboni
DD-WRT Novice


Joined: 04 Mar 2016
Posts: 49

PostPosted: Sat Feb 29, 2020 16:23    Post subject: Reply with quote
yoyoma2 wrote:
Sounds like a lot of work to keep a git repo up to date with BS's svn which changes rapidly. BS is very productive.

Couldn't we just create/share a few scripts and svn patches that setup the build environment.
    1) Start with one host eg. ubuntu 18.04 LTS
    2) Start with one target eg. Northstar
    3) Give the community build a name K2: Savage Mountain (not Kong 2 Wink)
    4) Share scripts/patches/toolchain(s)
    5) Try to get BS to submit useful features/fixes we come up with
BS is not unreasonable when presented with changes/fixes. Just from my suggestions he fixed cron, missing lines in syslog viewer, an error when stopping privoxy and even a Makefile only fix in iptables.

Having an easy way for the community to build will attract/keep highly skilled users improving dd-wrt benefiting all users.


I think that's a good plan overall (I'm actually starting on a similar list with a clean Ubuntu 18 VM and brand new checkout of DD-WRT), I just know how quickly the "shared scripts and patches" leads to a mess where you can't figure out which the right scripts / order / ... is. Maybe a halfway point would be a Git repo or at least Wiki for the scripts/patches?

Cheers!
kernel-panic69
DD-WRT Guru


Joined: 08 May 2018
Posts: 7178
Location: Texas, USA

PostPosted: Sat Feb 29, 2020 17:02    Post subject: Reply with quote
I would say limit it to Broadcom-only in general with only the kernels required - 2.4, 2.6, 3.10, 4.4(?), maybe 3.18 to keep the overall repo size down. I think quarkysg called it 'selective checkout' with SVN, limiting it to only the code required. Most public resources (github, bitbucket) limit the size of a repository to a certain size unless you pay $$$, if I am not mistaken. It doesn't have to be 'public', you can also limit access to specific users, if I am not mistaken. I would have to set up another separate dev environment altogether to do anything, anyway. Honestly, it doesn't matter what revision control system you use, maintaining it with as much activity goes on with DD development will be a full-time job. I'm open to whatever. Speaking of quarkysg, I ain't seen him around in a minute, I think we lost him to the temptation of OpenWRT?

rkboni wrote:
I think that's a good plan overall (I'm actually starting on a similar list with a clean Ubuntu 18 VM and brand new checkout of DD-WRT), I just know how quickly the "shared scripts and patches" leads to a mess where you can't figure out which the right scripts / order / ... is. Maybe a halfway point would be a Git repo or at least Wiki for the scripts/patches?

Cheers!


I wanted to address this in particular. What we could have is one person maintaining the git repo and all the collaboration done as pull requests with effective commentary so people hopefully aren't stumbling all over one another's work, perhaps Wink

_________________
Official Forum Rules, Guidelines, and Helpful InformationFirmware FAQInstallation WikiWhere Do I Download Firmware?
DON'T use Chromium-based browsersRTFM/STFW - TL;DR is NOT an excuse. • Why Should I Care What Color the Bikeshed Is?
Please DO NOT PM me with questions; Ask in the forum.

---------------------------------------------------------

Linux User #377467 counter.li.org / linuxcounter.net
yoyoma2
DD-WRT User


Joined: 24 Sep 2016
Posts: 146

PostPosted: Sat Feb 29, 2020 20:24    Post subject: Reply with quote
rkboni wrote:
Maybe a halfway point would be a Git repo or at least Wiki for the scripts/patches?
Sounds good. That way svn updates to BS's latest would be mostly effortless and if a patch conflicts, one person fixes the patch and all benefit. Kong must have had something similar set up but unfortunately never shared it.

github, gitlab, BitBucket, others... whoever takes the lead can choose.

Note that ubuntu 20.04 LTS is coming soon and will be supported for 5 years.

Attached a tweaked selective_co.sh that downloads one selected kernel only.



selective_co.sh.gz
 Description:
selective checkout with kernel

Download
 Filename:  selective_co.sh.gz
 Filesize:  561 Bytes
 Downloaded:  61 Time(s)

rkboni
DD-WRT Novice


Joined: 04 Mar 2016
Posts: 49

PostPosted: Sat Feb 29, 2020 20:34    Post subject: Reply with quote
UPDATED: got a little bit further, now it looks like I'm stuck due to a bad build of my OpenWRT toolchain... going to re-spin that, but not tonight. In the meantime adding patches for the initial setup and the Makefile bug I found as attachments.

hanskloss wrote:
Here's my experience compiling dd-wrt from sources for the ARM platform. Previous posts were very helpful in setting up the build platform. My post will focus more on steps and errors I have encountered rather than providing "do-it-all" scripts.
In the example below I have compiled a mini image for R7000 SVN revision: 33941 Full image is not recommended at first as too many changes are required, to make it work.
After taking the notes, I have re-installed the OS and pulled fresh dd-wrt sources to make sure these steps are reproducible.


Since there's been conversation about getting builds running again, I thought I'd start here with a clean Ubuntu 18.04 LTS VM and see how far I get...

hanskloss wrote:

PREPARATION


What toolchain?
I downloaded BS and Kong's firmwares with the same kernel and grep'ed files to see what they used.
Code:
strings /lib/libstdc++.so.6 | grep toolchain

BS result: toolchain-arm_cortex-a9_gcc-6.3.0_musl-1.1.16_eabi
Kong result: toolchain-arm_cortex-a9_gcc-6.3.0_musl_eabi

toolchain-arm_cortex-a9_gcc-6.3.0_musl-1.1.16_eabi is available in the dd-wrt.com downloads section
http://www.dd-wrt.com/site/support/other-downloads


This is clearly out of date for latest builds. Since I already had a GCC 9.2.0 toolchain build from OpenWRT's repo, I started off with that (toolchain-arm_cortex-a9_gcc-9.2.0_musl_eabi). This is probably also (slightly) out of date, as I built this at the end of Sept. 2019 to go with build r41027 (see https://forum.dd-wrt.com/phpBB2/viewtopic.php?p=1177650#1177650).

hanskloss wrote:

Linux distribution used: Debian Jessie amd64 8.9.0 DVD-1

Additional developer packages are needed to compile the mini version:
Code:
apt-get install subversion  build-essential ccache autoconf libtool cmake pkg-config bison flex gengetopt squashfs-tools lib32stdc++6



I used Ubuntu 18.04.4 LTS (server build), freshly installed. Added the same set of packages, which in my case mapped to the following versions:

subversion: 1.9.7-4ubuntu1
build-essential: 12.4ubuntu1
ccache: 3.4.1-1
autoconf: 2.69-11
libtool: 2.4.6-2
cmake: 3.10.2-1ubuntu2.18.04.1
pkg-config: 0.29.1-0ubuntu2
bison: 2:3.0.4.dfsg-1build1
flex: 2.6.4-6
gengetopt: 2.22.6+dfsg0-2
squashfs-tools: 1:4.3-6ubuntu0.18.04.1
lib32stdc++6: 8.3.0-6ubuntu1~18.04.1

hanskloss wrote:

Pull the sources:
Code:
svn checkout svn://svn.dd-wrt.com/DD-WRT -r33941



In my case, I grabbed -r42557, since that corresponded to the latest available build (or at least the latest build I found a "new build" thread for).

hanskloss wrote:

Export toolchain's bin into PATH

Code:
GCCARM=/home/user/toolchain-arm_cortex-a9_gcc-6.3.0_musl-1.1.16_eabi/bin
export PATH=$GCCARM:$PATH



Adjusted this for my newer toolchain.

hanskloss wrote:

During compilation, some packages complain about missing revision.h file. Create the files as below.

Code:
cd ~/DD-WRT/src/router/libutils
echo -n '#define SVN_REVISION "' > revision.h
svnversion -n . >> revision.h
echo '"' >> revision.h

cd ~/DD-WRT/src/router/httpd/visuals
echo -n '#define SVN_REVISION "' > revision.h
svnversion -n . >> revision.h
echo '"' >> revision.h

cd ~/DD-WRT/src/router/httpd
echo -n '#define SVN_REVISION "' > revision.h
svnversion -n . >> revision.h
echo '"' >> revision.h

cd ~/DD-WRT/src/router/shared
echo -n '#define SVN_REVISION "' > revision.h
svnversion -n . >> revision.h
echo '"' >> revision.h

cd ~/DD-WRT/src/router/services
echo -n '#define SVN_REVISION "' > revision.h
svnversion -n . >> revision.h
echo '"' >> revision.h

cd ~/DD-WRT/src/router/rc
echo -n '#define SVN_REVISION "' > revision.h
svnversion -n . >> revision.h
echo '"' >> revision.h



Copy the mini config
Code:
cd ~/DD-WRT/src/router
cp configs/northstar/.config_northstar_mini .config


Comment out CONFIG_SUPERCHANNEL SPUTNIK_APD CONFIG_HOSTAPD2 in the .config file as they are not used for Broadcom.

Cosmetic change. Comment out rules for non existant packages in the file ~/DD-WRT/src/router/rules/all.mk

Code:
#rules/all.mk:131: mmc-ixp4xx.mk
#rules/all.mk:214: strace.mk
#rules/all.mk:226: oled.mk
#rules/all.mk:237: ath9k-rules/ath9k.mk
#rules/all.mk:241: atm.mk



Minor updates here:
* strace.mk now exists, so left alone (line 217)
* oled.mk still missing (line 229)
* ath9k.mk moved to mac80211/ath9k.mk and exists, so left alone (line 240)
* atm.mk still missing (line 244)

hanskloss wrote:

Comment out Ralink drivers not used by our router, lines 287-296 in the Kconfig file
Code:
cd ~/DD-WRT/src/linux/universal/linux-4.4/drivers/net/wireless


#if RALINK_DEVICE
#source "drivers/net/wireless/rt3352/rt2860v2_ap/Kconfig"
#source "drivers/net/wireless/rt3352/rt2860v2_sta/Kconfig"
#endif
#if SOC_MT7620_OPENWRT
#source "drivers/net/wireless/rt7620/rt2860v2_ap/Kconfig"
#source "drivers/net/wireless/rt7620/rt2860v2_sta/Kconfig"
#source "drivers/net/wireless/rt5592/Kconfig"
#source "drivers/net/wireless/rt7612/rlt_wifi/Kconfig"
#endif


Proprietary Broadcom wireless modules
Make following corrections because of non existent source file to compile wl.ko module

cd ~/DD-WRT/src/linux/universal/linux-4.4/brcm/arm/wl/config
Comment out in the file wl.mk line 448 WLFILES_SRC_HI += src/wl/clm/src/wlc_clm_data.c


Note that none of the wlc_<stuff>.c files seem to exist (but pre-build objects do). Maybe the right thing to do is not to build anything in this directory? Still, commented out as per instructions.

hanskloss wrote:

cd ~/DD-WRT/src/router
Comment out lines 181 182 inside Makefile.northstar where it copies wlc_clm_data.c


Just to be safe, commented out all instances of the cp/touch pattern (lines 170-171, 173-174, 179-180, 182-183).

hanskloss wrote:

cd ~/DD-WRT/src/linux/universal/linux-4.4/drivers/net/wl/
Append following to the line 95 $(SRCBASE_OFFSET)/wl/clm/src/wlc_clm_data.o in the file wl_generic.mk


Edit emf.mk Warning! make files are space/TAB sensitive. Need TAB before each install -D command.
cd ~/DD-WRT/src/router/rules

Code:
emf:
# $(MAKE) -C emf/emfconf CROSS=$(CROSS_COMPILE)
# $(MAKE) -C emf/igsconf CROSS=$(CROSS_COMPILE)
# $(MAKE) -C emf/emf
# $(MAKE) -C emf/igs

emf-clean:
# $(MAKE) -C emf/emfconf clean
# $(MAKE) -C emf/igsconf clean
# $(MAKE) -C emf/emf clean
# $(MAKE) -C emf/igs clean

emf-install:
# $(MAKE) -C emf/igsconf CROSS=$(CROSS_COMPILE) INSTALLDIR=$(INSTALLDIR)/emf install
# $(MAKE) -C emf/emfconf CROSS=$(CROSS_COMPILE) INSTALLDIR=$(INSTALLDIR)/emf install
# $(MAKE) -C emf/igs CROSS=$(CROSS_COMPILE) INSTALL_MOD_PATH=$(INSTALLDIR)/emf install
# $(MAKE) -C emf/emf CROSS=$(CROSS_COMPILE) INSTALL_MOD_PATH=$(INSTALLDIR)/emf install
   install -D emf_bin/$(ARCH)/emf.ko $(INSTALLDIR)/emf/lib/modules/$(KERNELRELEASE)/emf.ko
   install -D emf_bin/$(ARCH)/igs.ko $(INSTALLDIR)/emf/lib/modules/$(KERNELRELEASE)/igs.ko
   install -D emf_bin/$(ARCH)/emf $(INSTALLDIR)/emf/usr/sbin/emf
   install -D emf_bin/$(ARCH)/igs $(INSTALLDIR)/emf/usr/sbin/igs


I needed to use serial recovery in my builds because included wireless modules would cause my router to "kernel panic" during the boot.
Replace emf.ko and igs.ko in the directory ~/DD-WRT/src/router/emf_bin/arm with modules which come from BS or Kong's firmware.

Good modules have the following sizes:
21864 emf.ko
16900 igs.ko

WebGUI
Remove optimizefiles from kromo/dd-wrt/Makefile line 1139


The original line here is
Code:
install: installwebfiles stripfiles $(PROCESSLANGFILES) optimizefiles removeobsolete makewww removepwc cleanup
and is now line 1168. Removed optimizefiles as per instructions.

hanskloss wrote:

Final web interface will not be fully functional. Some solutions exists: using tsynik patches or extracting javascripts from the www binary. All asp and javascripts page sources are compiled into single www binary
~/DD-WRT/tools/filelist contains the exact listing of all files(web pages and images)




CONFIGURE

These packages need to be compiled first in this order. Kernel step will generate kernel.release file which is later needed for compilation of other packages.

Code:
make -f Makefile.northstar jansson-configure
make -f Makefile.northstar jansson
make -f Makefile.northstar nvram
make -f Makefile.northstar utils
make -f Makefile.northstar install_headers 
make -f Makefile.northstar iptables
make -f Makefile.northstar libnet
make -f Makefile.northstar kernel



Fails on the last step:

Code:

make[1]: Leaving directory '/home/rafal/dev/DD-WRT/src/linux/universal/linux-4.4'
Makefile.northstar:166: recipe for target 'kernel' failed


due to:

Code:

  CC      arch/arm/kernel/bios32.o
In file included from arch/arm/plat-brcm/mpcore_core.c:25:
./brcm/arm/include/bcmutils.h:87:10: fatal error: hnd_pktq.h: No such file or directory
   87 | #include <hnd_pktq.h>
      |          ^~~~~~~~~~~~
compilation terminated.
scripts/Makefile.build:277: recipe for target 'arch/arm/plat-brcm/mpcore_core.o' failed


I'm not sure if this is relevant at this point, since it does produce the kernel.release file and so I continued on...


hanskloss wrote:

Fix the typo in the libubox.mk line 7. Include dir should be json-c not json

cd ~/DD-WRT/src/router/rules
UBOX_EXTRA_CFLAGS=-I$(TOP)/_staging/usr/include -I$(TOP)/_staging/usr/include/json-c $(COPTS) $(MIPS16_OPT)

In the previous revisions compiliation of the libubox package frequently failed. Run these steps too ensure it compiles fine.

Code:
make -f Makefile.northstar json-c-configure
make -f Makefile.northstar json-c
make -f Makefile.northstar libubox-configure
make -f Makefile.northstar libubox


Final configure and all
Code:
make -f Makefile.northstar configure
make -f Makefile.northstar clean all



Next, I tripped over an error compiling in src/router/rc like so:

Code:

make[1]: Entering directory '/home/rafal/dev/DD-WRT/src/router/rc'
[CC] rc.o
arm-linux-uclibc-gcc: warning: environment variable 'STAGING_DIR' not defined
In file included from /home/rafal/dev/DD-WRT/src/router/shared/netconf.h:24,
                 from rc.c:26:
/home/rafal/dev/DD-WRT/src/linux/universal/linux-4.4/brcm/arm/common/include/proto/ethernet.h:29:10: fatal error: packed_section_start.h: No such file or directory
   29 | #include <packed_section_start.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
Makefile:1074: recipe for target 'rc.o' failed
make[1]: *** [rc.o] Error 1
make[1]: Leaving directory '/home/rafal/dev/DD-WRT/src/router/rc'
rules/all.mk:339: recipe for target 'rc' failed


This looks like it might be a bug in the rc/Makefile, which this change should fix:

Code:

Index: rc/Makefile
===================================================================
--- rc/Makefile   (revision 42576)
+++ rc/Makefile   (working copy)
@@ -7,8 +7,8 @@
 
 
 ifeq ($(CONFIG_80211AC),y)
-CFLAGS   = -I. -I$(LINUXDIR)/brcm/$(ARCH)/include -Wall
-CFLAGS   = -I. -I$(LINUXDIR)/brcm/$(ARCH)/common/include
+CFLAGS   += -I. -I$(LINUXDIR)/brcm/$(ARCH)/include -Wall
+CFLAGS   += -I. -I$(LINUXDIR)/brcm/$(ARCH)/common/include
 endif
 
 ifeq ($(CONFIG_MSSID),y)


My next failure isn't very much further and appears to be related to how I built my OpenWRT toolchain. I'm calling it a night but have an idea of what needs fixing, so will follow up soon.

hanskloss wrote:

INSTALL


Compile trx in opt/tools and copy to src/router/tools
These files are for Netgear and D-link. Asus and TP-Link need some fixes so I skipped those.

Code:
cd ~/DD-WRT/opt/tools/
gcc -o trx trx.c
gcc -o trx_n trx_n.c
cp trx ../../src/router/tools
cp trx_n ../../src/router/tools


Create missing link to the opt folder
Code:
cd ~/DD-WRT/src/router
ln -s ../../opt opt


Original mksquashfs4 gives me illegal instruction error. I used the debian version which works fine.
Replace the line 575 in the Makefile.northstar
Code:
/usr/bin/mksquashfs $(ARCH)-uclibc/target $(ARCH)-uclibc/root.fs -comp xz -nopad  -root-owned -noappend -Xbcj arm


Since trx tools were not compiled for TP-Link and Asus comment out the following lines in the Makefile.northstar
583-588 for TP-Link
655-664 for Asus
668-669 for Buffalo

Final command
make -f Makefile.northstar install

Look for the firmware inside this folder:

~/DD-WRT/src/router/arm-uclibc



Some tips


R7000 Recovery:
During the boot ctrl+ C
At the CFE>tftpd
tftp -m binary 192.168.1.1 -c put K3_R7000.chk


Example of selectively restoring source folders:

svn checkout svn://svn.dd-wrt.com/DD-WRT/src/router/httpd -r 33941


Example of compiling single kernel module

GCCARM=/home/user/toolchain-arm_cortex-a9_gcc-6.3.0_musl-1.1.16_eabi/bin
export PATH=$GCCARM:$PATH
cd ~/DD-WRT/src/linux/universal/linux-4.4
make ARCH=arm modules SUBDIRS=drivers/net/switch

Displaying variables or paths during compilation:

Insert into Makefile.northstar following line as needed.
$(info ******************)
$(info My ARCH is $(ARCH))
$(info ******************)



fix-northstar-build-files.patch.gz
 Description:
Patch for initial source tree setup tweaks.

Download
 Filename:  fix-northstar-build-files.patch.gz
 Filesize:  1.49 KB
 Downloaded:  59 Time(s)


rc-makefile.patch.gz
 Description:
rc/Makefile bugfix

Download
 Filename:  rc-makefile.patch.gz
 Filesize:  236 Bytes
 Downloaded:  60 Time(s)



Last edited by rkboni on Sun Mar 01, 2020 4:17; edited 2 times in total
kernel-panic69
DD-WRT Guru


Joined: 08 May 2018
Posts: 7178
Location: Texas, USA

PostPosted: Sat Feb 29, 2020 20:53    Post subject: Reply with quote
@yoyoma2, it should be limited to only what is required for Broadcom platforms only, which may take some investigation, but that is a good start. You, egc, and rkboni probably need to compare notes and such.

@rkboni, if you haven't been following svn commits, you may run into more issues than what hanksloss annotated. There have been plenty of changes to Broadcom since his write-up.

Also, you should likely keep from choosing a specific revision to checkout, save and except for specific circumstances which will require back-tracking. It would be great if we had a different common place to discuss, but it is what it is at this point. I may look into setting up a github. What would be ideal is to clean out all of the unused cruft from the source tree so we are only looking at what is absolutely necessary, but that is semi-optional. The ideal goal would be to be able to submit patches to fix all Broadcom builds, not just Northstar. Anyhow, some more of my $0.02 on this.

_________________
Official Forum Rules, Guidelines, and Helpful InformationFirmware FAQInstallation WikiWhere Do I Download Firmware?
DON'T use Chromium-based browsersRTFM/STFW - TL;DR is NOT an excuse. • Why Should I Care What Color the Bikeshed Is?
Please DO NOT PM me with questions; Ask in the forum.

---------------------------------------------------------

Linux User #377467 counter.li.org / linuxcounter.net
rkboni
DD-WRT Novice


Joined: 04 Mar 2016
Posts: 49

PostPosted: Sat Feb 29, 2020 22:08    Post subject: Reply with quote
kernel-panic69 wrote:
@yoyoma2, it should be limited to only what is required for Broadcom platforms only, which may take some investigation, but that is a good start. You, egc, and rkboni probably need to compare notes and such.


Given the rather opaque build structure and amount of code in the tree, I suspect that getting to "what is required for Broadcom platforms only" is a huge goal. But it would be nice not to have to drag all that code around!

kernel-panic69 wrote:

@rkboni, if you haven't been following svn commits, you may run into more issues than what hanksloss annotated. There have been plenty of changes to Broadcom since his write-up.

Also, you should likely keep from choosing a specific revision to checkout, save and except for specific circumstances which will require back-tracking. It would be great if we had a different common place to discuss, but it is what it is at this point. I may look into setting up a github. What would be ideal is to clean out all of the unused cruft from the source tree so we are only looking at what is absolutely necessary, but that is semi-optional. The ideal goal would be to be able to submit patches to fix all Broadcom builds, not just Northstar. Anyhow, some more of my $0.02 on this.


Oh, I'm sure there are more issues than annotated by hanksloss... But the SVN commits haven't been particularly instructive in terms of what's changing and particularly how it affects the builds. Part of my starting there was because it was a known point where things did actually build, and the other part was because others seemed to be starting there as well.

Re: checking out a specific version of SVN, I agree in principle... but given that we're way behind the 8-ball in terms of getting anything close to a sane build, I figured starting at a revision that's been known not to have build issues (because someone built it and people are using that build) would at least save some potential aggravation with trying to track a moving target. Once we get some revision building, then we can work on expanding the scope.

At least we now have a group of interested folks talking about it -- that's a great start. Any contributions, be they better ways to coordinate or communicate about this, pointers to other useful scripts/info, etc. are always welcome!

Cheers!
dominic0315
DD-WRT Novice


Joined: 21 Apr 2020
Posts: 10

PostPosted: Thu Apr 23, 2020 1:19    Post subject: Reply with quote
Toolchains

I wonder how important sticking to a compiling chain is (considering newer GCCs) to the consideration whether a workable image could be obtained. It may be due to some specific flags passed when building the cross-compiler to ensure low-level logics not optimised away probably. But my point is, I think for a better source tree, we probably have to think of how to eliminate all these non-generic elements in the repository. And that the repository should just reside a auto build script that checkouts GCC source tree and compile the toolchain in user environment with a `make ARCH=arm LIBC=musl cc`.

Leveraging Subtrees or Other Mechanism

My gut feeling would be we have to eliminate the Linux source trees and other packages source trees if we are to establish a new repository. The new repository would probably just include references to the relevant repositories and at the same time have reference to specific checkout that are known to be stable from upstreams e.g., the Linux source tree.

Other than these references, the repository then keeps the relevant patches or extra low-level header files, source files created specifically for some devices that are supposed to overlay to the upstream source trees. And once these patches are applied up-stream, they should be removed in the newer commits. We may need handle special cases where,
1) user requires earlier package version; DD-WRT patches already submitted upstream and eliminated from the current source tree.
2) user requires experimental (unstable) version that includes DD-WRT patches submitted upstream; DD-WRT stable version still contains the patches as the referring stable package version has not been including the patches.

We probably should diff-ing all the "external" source trees and find out how Kong and BS worked on the "external" source trees and establish relevant policies to how our tree should overlay to the "external" source trees.

My expectation would be either using menuconfig for users to select whether building DD-WRT taking defaults (all stable packages and kernel), e.g.,

Code:
make config
Are you to build the DD-WRT firmware with known stable kernel and packages? [Y/n]: n
Are you to build the DD-WRT firmware with recognised stable Linux 4.4.208? [Y/n]: y
Are you to build the DD-WRT firmware with recognised stable package bird [commit xxxxxxx] [Y/n]: n
Please specify the commit to check out for building bird: yyyyyyy
...


Checking

We probably need more autoconfig scripts to ensure pre-requisites are met for compiling DD-WRT source tree.

Separation of Generic Code and Device Specific Configs

We also need to establish some mechanism to ensure reuse of generic attributes. Probably tuple-based config files that says a specific device should supports some features and should compile a certain package; or that a device should apply certain compilation FLAGS etc. By then, if a user issue `make EA6900` would build a stable version of binaries from DD-WRT source tree, that also checkouts all recognised stable versions of packages and Linux builds.

E.g.,

device.arch.conf
ea6900 : armv7 vfp3
e2000 : ...

I am interested in working alongside many of you if we are to establish a new source tree.
kernel-panic69
DD-WRT Guru


Joined: 08 May 2018
Posts: 7178
Location: Texas, USA

PostPosted: Thu Apr 23, 2020 2:09    Post subject: Reply with quote
There are already people collaborating, and what you're suggesting would completely stop their work and cause unnecessary headaches. Their goal is to submit patches to the main developer for everyone's benefit. What you're suggesting may linger in the grey areas of violating the GPL and might get you in trouble - and we don't need any more of that here. I am going to leave things open and your post up - for now.
_________________
Official Forum Rules, Guidelines, and Helpful InformationFirmware FAQInstallation WikiWhere Do I Download Firmware?
DON'T use Chromium-based browsersRTFM/STFW - TL;DR is NOT an excuse. • Why Should I Care What Color the Bikeshed Is?
Please DO NOT PM me with questions; Ask in the forum.

---------------------------------------------------------

Linux User #377467 counter.li.org / linuxcounter.net
dominic0315
DD-WRT Novice


Joined: 21 Apr 2020
Posts: 10

PostPosted: Thu Apr 23, 2020 2:15    Post subject: Reply with quote
I see! Which means it's more like fixing the current SVN repositories than to have a revamp of the code base.

By the way, do you mind educating me a bit on the violation of GPL Licence point?

Thanks very much!
kernel-panic69
DD-WRT Guru


Joined: 08 May 2018
Posts: 7178
Location: Texas, USA

PostPosted: Thu Apr 23, 2020 2:46    Post subject: Reply with quote
Well, as far as I know, this project is still under v2 of the gpl: https://www.gnu.org/licenses/old-licenses/gpl-2.0.html

But, of course, we are now in the days of v3: https://www.gnu.org/licenses/gpl-3.0.html

I am not saying it would be a violation to do what you propose. But the changes you propose would have to be publicly published and license granted. This is not a black and white subject here, unfortunately. Only one person who could explain the issue, honestly. Also, it would be a very tedious and long process to do what you propose. Good luck with that.

_________________
Official Forum Rules, Guidelines, and Helpful InformationFirmware FAQInstallation WikiWhere Do I Download Firmware?
DON'T use Chromium-based browsersRTFM/STFW - TL;DR is NOT an excuse. • Why Should I Care What Color the Bikeshed Is?
Please DO NOT PM me with questions; Ask in the forum.

---------------------------------------------------------

Linux User #377467 counter.li.org / linuxcounter.net


Last edited by kernel-panic69 on Thu Apr 23, 2020 16:33; edited 1 time in total
rkboni
DD-WRT Novice


Joined: 04 Mar 2016
Posts: 49

PostPosted: Thu Apr 23, 2020 2:52    Post subject: Reply with quote
dominic0315 wrote:
I see! Which means it's more like fixing the current SVN repositories than to have a revamp of the code base.


What you're describing is very much the OpenWrt model where most of the code lives external to the repo and and just package lists and generic build scripts handle most of it. I'm pretty sure they still have their own repos for some things but generally their builds are driven off of external sources, local patches and local build config.

DD-WRT trades that for an all inclusive source tree that also includes vendor firmware / binary driver blobs and thus supports lots more hardware.

Different models for different folks.. while I admire the simplicity of OpenWrt it's never supported much of my router hardware, which is why I've been running DD-WRT forever. But whichever you run, you're unlikely to be able to turn one project into the other.
kernel-panic69
DD-WRT Guru


Joined: 08 May 2018
Posts: 7178
Location: Texas, USA

PostPosted: Thu Apr 23, 2020 3:14    Post subject: Reply with quote
rkboni wrote:
dominic0315 wrote:
I see! Which means it's more like fixing the current SVN repositories than to have a revamp of the code base.

What you're describing is very much the OpenWrt model

.....

you're unlikely to be able to turn one project into the other.


Without a LOT of tedious work. Ain't gonna be drag-n-drop. And that modification alone to the overall picture is the bane of the grey area of the GPL earlier referenced.

_________________
Official Forum Rules, Guidelines, and Helpful InformationFirmware FAQInstallation WikiWhere Do I Download Firmware?
DON'T use Chromium-based browsersRTFM/STFW - TL;DR is NOT an excuse. • Why Should I Care What Color the Bikeshed Is?
Please DO NOT PM me with questions; Ask in the forum.

---------------------------------------------------------

Linux User #377467 counter.li.org / linuxcounter.net
dominic0315
DD-WRT Novice


Joined: 21 Apr 2020
Posts: 10

PostPosted: Fri Apr 24, 2020 1:21    Post subject: Reply with quote
rkboni wrote:
What you're describing is very much the OpenWrt model where most of the code lives external to the repo and and just package lists and generic build scripts handle most of it. ...

DD-WRT trades that for an all inclusive source tree that also includes vendor firmware / binary driver blobs and thus supports lots more hardware.

... while I admire the simplicity of OpenWrt it's never supported much of my router hardware, which is why I've been running DD-WRT forever. ...


I believe it's rather irrelevant to whether the reliance on external source trees "OpenWRT" model traded support but rather it is OpenWRT's philosophy to insist their compiled binaries must be based on Open Source Softwares that hinders its support on from their compiled binaries.

Lack of device support is also the reason why I started to use DD-WRT, especially with Broadcom devices that cannot go beyond 802.11g . But purely out of academic discussions, I believe it would be do-able DD-WRT taking the "OpenWRT" source control model while continuously support more and more devices as long as this project is not following OpenWRT rigidity in compiling only OSS, i.e., supporting proprietary blobs in the source tree.

But that said, I wonder if having a side-by-side "moderated" repository that occasionally synchronise codes with external sources would benefits the wider community by generating personalised firmwares based on "generic" source tree. This "moderated" repository and the "developer" DD-WRT would be something analogous to RedHat and Fedora, or CentOS and Fedora relationship.

And by the way, I believe I read something earlier that BrainSlayer was trying to merge/sync the code with OpenWRT but I wonder what is the scope or how it goes?
Goto page Previous  1, 2, 3 ... 15, 16, 17, 18  Next Display posts from previous:    Page 16 of 18
Post new topic   Reply to topic    DD-WRT Forum Forum Index -> Generic Questions All times are GMT

Navigation

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You can attach files in this forum
You can download files in this forum