The best open source alternative to GitBook is BookStack, which is both free and Open Source. If that doesn't suit you, our users have ranked more than 25 alternatives to GitBook and many of them is open source so hopefully you can find a suitable replacement. Other interesting open source alternatives to GitBook are DokuWiki (Free, Open Source), Read The Docs (Free, Open Source), Docusaurus (Free, Open Source) and mdBook (Free, Open Source).
ditto inspired by flatdoc is a markdown code documentation system, capable of producing documentation similar to Three.js and Backbone.js, check out live demo here. It is designed for people who use Github Pages to host their code documentation, but you can also host it yourself.