Repository navigation
Code SplittingΒ #7
Description
GWT has a cool feature named Code Splitting which loads parts of the code on demand. This is also appliable for webmake.
Here my idea:
// a.js:
var b = require("b");
// ...
bindEventHandlerForSomeRarEvent(function() {
// the magic keyword "process.nextTick", which behaves normally in node.js
// but so some special thing in webmake
process.nextTick(function(err) {
// Some code loading err handler
// err is undefined in node.js
if(err) { /* ... */ }
var c = require("c"); // really big libary, which is only needed in this rar case
// ...
});
});webmake core would provide the magic process.nextTick function to the compiled code.
process.nextTick do this in the compiled code:
- store the callback function
- adds a script tag to the document
- the script contains JSONP
- callback function (JSONP) adds some new modules to the context
- and calls the stored callback function.
- in case of an error (ex. timeout) the stored callback function is called with some kind of error parameter
Here is a quick hint on the compiled files:
// output.js
( /* core code */
({
"a": function(/* ... */, process) {
// a.js:
var b = require("b");
// ...
bindEventHandlerForSomeRarEvent(function() {
// the magic keyword, which behaves normally in node.js
// but so some special thing in webmake
process.nextTick(function(err) {
// Some code loading err handler
// err is undefined in node.js
if(err) { /* ... */ }
var c = require("c"); // really big libary,
// which is only needed in this rar case
// ...
});
});
},
"b": function /* content of b.js */
// here is no "c": !!
// which saves bandwidth// may be named "a.output.js":
webmake_magic_jsonp_function( // the jsonp callback function
"a", // origin of code loading (used for identifing the stored function)
{
"c": function(/* ... */, process) {
// the content of the big libary
},
"d": function(/* ... */, process) {
// d.js is here a dependency of c.js (optional)
}
}This feature may makes the compile process a big complexer.
It may be useful for:
- loading parts of your webapp when accessing them
- loading polyfills only when necessary
- decrease startup time
Reacted by Aizen M, qianxuemin, Piyush Bansal, Abhay Gupta, Abdallah Hemdan, Dhruv Kothari, Ashish Duklan, Leone.Y, Alexander Kazakov, MaksAnn and 3 more
Reacted by Vision_X, Marenda SA and Kevin Lai
Reacted by Artem Sapegin, Haroen Viaene, chee, Dhruvdutt Jadhav, Daniele, HuΓ‘ng JΓΉnliΓ ng, Mohsin, Aidan B, Ahsan Naveed, Cheng Hu and 26 more
Reacted by TUARAN, Alexander Myshov, εΈη, Zixuan Chen, Abhay Gupta, Abdallah Hemdan, Dhruv Kothari, Ashish Duklan, Nivesh Birangal and Kevin Lai
Reacted by Abhay Gupta, Artem Savelev and Dhruv Kothari
Metadata
Metadata
Assignees
Labels
No labels
Activity
medikoo commentedon Feb 26, 2012
Code splitting is indeed most missing major feature and I plan to add it with next big update.
However in first place I imagine it just as a possibility of creating few different bundles that reuse same modules which are loaded once with one of the bundles, that's it.
I don't think it's good idea to change semantics of
process.nextTickfor client-side for that. If we would really need some function to do async modules bundle load on client side it should be specific to client-side and be used in code written specifically for client-side.Phoscur commentedon Feb 26, 2012
Isn't
setTimeout(fn, 0)the browser replacement forprocess.nextTick(fn)?Phoscur commentedon Feb 26, 2012
Maybe this is the problem, you should not include a big library if you don't really need it.
I'm fine with having a little more startup time and then a more fluent interface. I would like to have an option in the opposite direction, so that module code can be preexecuted to be ready when it is required, but that's probably not worth the effort.
This does not mean that I don't like the idea of sending code later to the client, I see usecases for that. But it's not to reduce startup time. It would be nice to be able to rollout new features without a whole app refresh.
sokra commentedon Feb 26, 2012
@medikoo
i don't think this changes the semantics of
process.nextTick.process.nextTickexecutes some code in the near future in node.js. It you implement Code Splitting as i propossed it has the same semantics. It executes some code in the near future. There is a code loading in between on client side, but this is transparent to the user.@Phoscur yes
setTimeout(fn, 0)is the replacement, but i don't want to use a existing function (on client side) because this could impact exisiting code. I thinkprocess.nextTickis the best choice because it has nearly no impact on node.js. The only impact is that the code is executed later. This semantic stays the same on client side (Ok, the delay is bigger on client side)Consider a big webapp with several features. The features are in subpages, but because you do not want a whole page reload you have all features in one codebase.
Without Code Splitting this has a negative impact on startup time because you compiled your codebase with webmake into a big js file.
To optimize, you want to split your code and you only want to load if you navigate into the subpage. So insert
process.nextTicksomewhere in your code (i should be easy in ansynchronous functions). Now your compiled codebase will be splitted into pieces and only the first piece is loaded on webapp startup. This decreases startup time in compare to webmake without this feature.This feature is optional because you don't need to use
process.nextTick, but if you decide to use it it is transparent to the programmer.Phoscur commentedon Feb 26, 2012
Why wouldn't you want all features in one codebase?
You can pack about 300k codelines into one megabyte minified scriptfile, how big is your webapp going to get? I think the overhead to request a few bytes when you need them is bigger than a little slower loading time in the beginning.
Keep in mind that the code is only loaded and parsed in the beginning, it is lazy-executed when it's needed.
Creating some shims for node specific functions like process.nextTick is not a bad idea, we can probably steal those from somewhere else anyways.
medikoo commentedon Feb 29, 2012
@sokra I would definitely not mess with
process.nextTickit's Node specific and was not intended for asynchrounous background loading.Anyway as I mentioned, soon we'll have the way to create many separate bundles that share modules and load them on demand. It can be helpful when you have really big application with clearly separated functionalities.
sokra commentedon Mar 8, 2012
I reimplemented webmake from scratch to reduces some limitations of it and add code splitting.
It is just a start and was made in a 2 day hacking session.
It do need some additional love but the most things are working really good.
See https://github.com/sokra/modules-webpack for some docs
and https://github.com/sokra/modules-webpack/tree/master/example for a example
Here is a list of differences to the original webmake:
Module and file names are replaced with a unique id per file
=> different versions of the same module do not overlap anymore
=> no need for resolving on client side => smaller
It is allowed to require outside to current package scope
Browser replacements
web_modulesis preferred tonode_modulesxxx.web.jsis preferred toxxx.jsifrequire("./xxx")without extensionCode splitting with
require.ensureas in http://wiki.commonjs.org/wiki/Modules/Async/A
see links above
medikoo commentedon Mar 9, 2012
@sokra thanks for input.
Your approach is very interesting, but you introduced big limitation with that. You didn't take into account specific cases when modules are not required directly.
In code you may pass generated string to require or run require functions indirectly. Such require calls of course cannot be picked statically but in
webmakethere'sincludeoption through which you can force to include such modules, so there's back door to deal with that, in your fork I guess it's not possible.In projects I currently work with this is the way templates are loaded, indirectly by template engine, I wouldn't be able to use your solution there.
Other goal, when working on webmake, was not to change source code of modules, it's not that major concern as one above, but still it's certainly better for development and debugging to keep module names and paths intact.
If you decide to continue development of your fork please change it's name, as you pointed it's very different solution.
Thanks!
sokra commentedon Mar 9, 2012
@medikoo thanks for your feedback
The limitation, you pointed out, is a problem, but i think a "back door" can be added with some config options. Do you have a example of a use case? Can you post it?
The changes in the source code are really small so i don't think that this is a big problem.
The point of posting it here is a aspect of code splitting which may can used in your webmake (also without the other changes). It's the same approach as in the inital post, but with
require.ensureinstead ofprocess.nextTick. (As you argued correctlyprocess.nextTickis more a node specific thing)I'll rename it if you want so. It was orginally intended as proof of concept for this issue.
medikoo commentedon Mar 9, 2012
@sokra I'm not sure if any back door is available in your solution. Code can only be scanned statically, but all requires should work programmatically, so e.g. you should provide some sort of back door, so following could work:
When you do all paths resolution on server side, and for client you don't provide any mechanism to do real path resolution then it's no place for back door.
For example here https://github.com/medikoo/domjs/blob/master/lib/domjs.js#L92 I call require indirectly.
sokra commentedon Mar 9, 2012
Hmm... this really does not seem to work. This stays a limitation.
What do think about the code splitting?
medikoo commentedon Mar 10, 2012
To have code splitting, in first place we need to add possibility of creating different bundles that share same modules, form that point you can upload such bundles on demand in many different ways.
By example, code of additional bundle that we may need to load, will probably look like:
Where
requireis global variable exposed by bundle that was initially loaded.I'm still not sure do any load on demand functionality should be included in
webmake, or should it be left to external tools how to handle that. We'll figure it out in near future.sokra commentedon Mar 10, 2012
I've moved it to https://github.com/webpack/webpack
Hope that is ok...
medikoo commentedon Mar 10, 2012
That's ok, thanks
sokra commentedon May 7, 2012
A few moths ago, I worked much on webpack and it's working really good.
Here is a example web app: http://webpack.github.com/example-app/
sokra commentedon Oct 22, 2012
I don't need this issue anymore. Should I close it or do you want it to stay open?
medikoo commentedon Jun 23, 2013
I plan to add this feature, with Webmake v0.4, so I'll keep it open until it's done.
medikoo commentedon Dec 11, 2016
@JHGitty nothing yet, but you may expect something 1st quarter next year.
abhinavsingi commentedon Dec 15, 2017
So that's how it all started!
https://twitter.com/TheNicoKoenig/status/941410877290491905
BhargavaCreates commentedon May 1, 2021
Like seriously! this is how Webpack was born π @sokra
WileyPro commentedon Jul 8, 2021
wow!
Ashish2097 commentedon Dec 14, 2021
Genesis π
jjkbb123 commentedon Sep 29, 2022
η«ι³εζ
ap221882 commentedon May 1, 2023
Happy to see the history in a thread.
xuejiangping commentedon Aug 23, 2023
hello webpack
Suhas010 commentedon Mar 16, 2025
interesting!