dispatcher.lsapi load all controllers

[fantasydreaming]

I haven't done very thorough testing, but I believe that the dispatcher.lsapi loads the rails framework, then spawns the children, correct?

It'd be cool if it could also go through my controllers directory and load each controller - as that's another 3 seconds of intensive file loading, and it would be greeeeat to have that memory shared across the child processes.


1) Where is the dispatcher.lsapi loaded from in the default configuration? How would I replace it with a customized version?

2) Will iterating in ruby over each *_controller file and requiring it work the way I expect it will with regards to shared memory?

3) On an unrelated note, I have problems sometimes with my app hanging on stupid things I'm doing in ruby - like waiting forever for a memcache socket to respond. These processes are then usually not killed by lsws, because they need kill -5. Any plans or suggestions how to get lsws to escalate the kill amount if children are not dieing?


Thanks for a great project!

 

[mistwang]

Answers to your questions:
If you want your controllers initialized before spawning children process, you should do it in environment.rb or something will be called during framework initialization, it will not give you shared memory automatically though.

1) Easy configuration uses lsws/fcgi-bin/RailsRunner.rb, dispatch.lsapi is a context created internally, and you should not customize it if you are not sure what is it doing.

2) If you controllers uses shared memory, then memory will be shared, otherwise, each process will get its own copy.

3) Please check the README file in ruby-lsapi 2.1 release, we already add some environment variables for this purpose.

 

[fantasydreaming]

Thanks, I'll try adding controller loading to my environment.rb.

1) Any way to verify how much memory is actually getting shared? Currently I'm trying to tell by watching the strace of the newly forked processes, but that's sketchy and the timing is difficult.

2) What do you mean by "uses shared memory". Do you just mean things that are loaded in the environment?

3) In the lsapi readme, I see some vars for LSAPI_MAX_IDLE_CHILDREN, which is interesting, but nothing directly memory related.

4) Would LSAPI_PGRP_MAX_IDLE help my problems with the parent not killing off the children with a high enough priority? Seems more related to the parent quitting when the children crash or something?

 

[mistwang]

1) command "ipcs" can list shared memory blocks allocated system wide. MMAP memory can be shared among parent/children processes as well.
2) No, the application explicitly uses shmxxx() or mmap() functions to allocation shared memory. Loading in environment will not use shared memory automatically.

3) those variables are not memory related, but procoess management related.

4) LSAPI_MAX_PROCESS_TIME will stop hanging children processes you described. LSAPI_PGRP_MAX_IDLE will stop the whole application incuding the parent process when it is in idle state too long, the rails framework have to been reinitialized when next request comes in.

 

[fantasydreaming]

1) ipcs -p [parent process] or [child process] doesn't list any memory as shared...

2) Any way to do this myself from within rails in my environment? My processes seem to grow to around 200 megs, so it'd be nice to share more of that initial ~70 megs of memory if I can.

4) Is just to save memory usage/etc on very seldomly used servers? Similar to turning 'load a startup' to off...

 

[mistwang]

1) then no shared memory is used.
2) Probably not, it is pretty low level, and is upto the developer of each component in Rails, you'd better spend sometime to find a way to optimize your application to use less memory.
4) Yes, that's correct.

 

[fantasydreaming]

Perhaps we're talking about different things when we say 'shared memory'.

I'm referring to the fact in linux that if a parent process loads something into memory, then forks 3 times, each child will 'share' the parents memory until that area gets dirtied by the child.

This is the point of having the parent process fork all the sub-processes, correct?

As far as I know, there's no easy way to tell if that memory is still shared or not.