Soldermask is seldom applied by silkscreen in industrial applications. The two most common masks are LPI (liquid photoimagable) and dry film - both are photoetched in a manner similar to the copper layers, but using different chemical processes.
Look for references describing "SMOBC" (solder mask over bare copper), "LPI", and "dry film" using a search engine. There is a lot of information on the web.
Your question about "assembly layer" is a mystery. There is no such layer in standard PCB layup terminology. Do you mean "overlay", "silkscreen", or "legend"? These are the top and bottom information layers that print the component ID and/or description on top of the mask layer. If that is what you are asking about - no, there is no practical substitute for silkscreen when it comes to printing the component information on the board.
If you are trying to find a way to do the mask and silkscreen at home, there is no easy way to do-it-yourself.