went from 512mb to 1gb
_________________
Pre-order you new KGB-Racing Shirt!
"Super Karate Monkey Death Car"

steve k. wrote:

went from 512mb to 1gb

thx,,, \8-)
_________________
www.jennaishealed.org ........ please visit my "other" baby... shes for sale!

woot!!



the swap isn't even being touched!!

now i just have to worry about logs filloing up the HD space. They keep resetting my slow.query setting.

steve k.
_________________
Pre-order you new KGB-Racing Shirt!
"Super Karate Monkey Death Car"

1gb? That still seems small. I am used to my 2850s with 4gb of RAM :P

Anyway, create a bash script to nuke those log files once a week.

I can help you if you want.

What version of mysql we running? What does the my.cnf look like? You can streamline that... and I can also build you apache 2.2 rpm (assuming you use redhat) as well. It will really fly then. :)
_________________
-Justin
--
'74 tii
'76 02
'97 m3
http://www.bmw2002.net

the issue with logs is that fior some reason the setting for the slow.query one gets reset every time we reboot the box. A little strange. So it logs every query run without an index. There are lot of them.. All i want is queries that take more them 5 seconds. And those practically do not exist.

steve k.
_________________
Pre-order you new KGB-Racing Shirt!
"Super Karate Monkey Death Car"

Whats the /etc/my.cnf look like? Keep in mind that my.cnf does NOT get installed with mysql... so if you set something via mysql command line and restart the service or the box you will lose the setting.

Inside my.cnf... (among other things):
You could change the value of:

long_query_time=6 and slow queries will be greater than 6 seconds.

Also you can set the location:

log_slow_queries=/var/lib/mysql/slow-queries.log

Also when you do a "mysqladmin", it will tell you what places it is reading the my.cnf from by default.

What version of mysql are we running? 5.0.something ?? 5.0.26 is current.
_________________
-Justin
--
'74 tii
'76 02
'97 m3
http://www.bmw2002.net

I do the same thing with my logs but every hour actually! We get about 500,000 page churns a day so we need it. We also need 4 machines with 16 gigs of memory each because we are running an enterprise Java application. No PHP kiddies here ;)
_________________

Yea you need 16GB of ram just load JVM itself ... thats without any programs :)

PHP here. All I can say is LAMP rules. We are at 40-50 million searches a day on one of our products.
_________________
-Justin
--
'74 tii
'76 02
'97 m3
http://www.bmw2002.net

Dont get me wrong. I also use PHP, but mostly only for quick and dirty things and personal projects. I dont think I would like to build a large e-commerce site on PHP again after knowing Java really well.

I also find that PHP allows developers to be extremely slopping and write one-use only code. That being said, anyone can write bad code, I just find that PHP structure lends itself to that.
_________________

_z_ wrote:

woot!! the swap isn't even being touched!!

but.. there's 1 zombie!! AHHHHH.

Man, at my new job there's a solaris box with 1500day uptime.
Bind quit on it and I had to log in to see what happened. .booya.

slash02 wrote:

_z_ wrote: woot!! the swap isn't even being touched!! but.. there's 1 zombie!! AHHHHH. Man, at my new job there's a solaris box with 1500day uptime. Bind quit on it and I had to log in to see what happened. .booya.

its holloween.. there's always one zombie!!

it was there just for a sec while I did the screen grab.

Steves controlling my.conf looks pretty groovy to me.

join_buffer_size=1M
key_buffer_size=50M
max_allowed_packet=128M
max_connections=500
max_connect_errors=10000
open_files_limit=65536
read_buffer_size=1M
sort_buffer_size=1M
table_cache=500
thread_cache_size=20
wait_timeout=200
long_query_time=5

Might be time to up the key_buffer_size since the new memory ..

sorry steve for exposing the guts.. its like letting someone look in your engine uninvited.

i'll quit

i don't know nuthin about any of that stuff. But I do know we need to decapitate or scramble the brains of that zombie, before we have more zombies.

If you guys arent doing any caching I would highly recommend it. We see have seen vast improvements while using EhCache wiht Hibernate. You could easily cache the last few hundred posts and reduce load on mySql 10-fold.

For mySimon.com we are keeping a cache for 30 minutes for all the editor generated content (blogs) which literally reduces the traffic to a few queries per half hour (sometimes I have seen multiple hits at high load due to cache refreshing I think?).
_________________

and was the server that thing i slept next to and was too scared to touch?

johnhup wrote:

If you guys arent doing any caching I would highly recommend it. We see have seen vast improvements while using EhCache wiht Hibernate. You could easily cache the last few hundred posts and reduce load on mySql 10-fold. For mySimon.com we are keeping a cache for 30 minutes for all the editor generated content (blogs) which literally reduces the traffic to a few queries per half hour (sometimes I have seen multiple hits at high load due to cache refreshing I think?).

thats some interesting stuff.

Just reading
http://ehcache.sourceforge.net/documentation/hibernate.html

This seems like a jre deal ya? I don't see how to integrate that into the php-mysql end of things on this system -- but since I'm mostly clueless thats not surprising :)

This site uses pretty bog standard php mysql queries .. it isn't clear how you could get ehcache + hibernate to work without rewriting the data methods for the whole thing.

Steve K has a beta site thats up so we could play around with some funky stuff without impacting the main site so it'd be worth checking out.

Got any docs or links about how to implement the cache ? I'm searching too.

I'm also curious about how to cache queries when the database tables are updated so often.. like if one person inserts a new post all the cached queries would be outdated. I'm sure the folk who write query caching have that one figured out but just wondering how that is handled.

cheers mate -- something new to me !

-zachary

Yes EhCache is a specific to Java, but PHP has lots of caching ability. EhChache works by caching an object in memory at the time it is first queried. In this use case I would be caching individual posting, not the entire thread. The thread is just a collection of postings. Then what I can do is set the time-to-live on those objects. Even making the time to live 1 minute can greatly reduce load on a database.

Other groups at my company also SQUID cache. This can run side-by-side with an Apache daemon, but preferably run a separate machine. It can even sit between the Apache box and the mySql box to cache queries. Its dumb, cheap, and extremely reliable (like my 2002 ;). Less moving parts.
_________________

[quote="_z_"][quote="slash02"]

_z_ wrote:

woot!! its holloween.. there's always one zombie!! it was there just for a sec while I did the screen grab. Steves controlling my.conf looks pretty groovy to me. join_buffer_size=1M key_buffer_size=50M max_allowed_packet=128M max_connections=500 max_connect_errors=10000 open_files_limit=65536 read_buffer_size=1M sort_buffer_size=1M table_cache=500 thread_cache_size=20 wait_timeout=200 long_query_time=5 Might be time to up the key_buffer_size since the new memory .. sorry steve for exposing the guts.. its like letting someone look in your engine uninvited. i'll quit

Looks good, and yeah, i thought that zombie was quite fitting.
up the buffer size, considering that your ram usage is pretty good. Probably want to MRTG it or something just to get an idea of usage over time though

We've had good luck with eaccelerator.

http://eaccelerator.net/

I'd start with trying to improve the php performance.
_________________
-Justin
--
'74 tii
'76 02
'97 m3
http://www.bmw2002.net

jgerry2002 wrote:

We've had good luck with eaccelerator. http://eaccelerator.net/ I'd start with trying to improve the php performance.

now that looks like something taisty that could be bolted on and tested for the beta site. Keeping compiled scripts in memeory and executing from there might make a huge difference -- especially since steve put in some extra memeory.

The main overhead was in the database before -- mostly keeping all those archived messages in active searches I think was the issue -- that and all the connections from an enthusiastic 2002 crowd!

Let me know I can give you some baseline settings for the php accelerator.

Yes it can do a very good job speeding things up.
_________________
-Justin
--
'74 tii
'76 02
'97 m3
http://www.bmw2002.net

See now I am going to have to crack a PHP joke ;). Java make take more overhead but performs ten times better once its running! The best I have seen PHP do is server 2-3 pages a second without caching! That in unacceptable in a production environment.

I couldnt help myself. Im a nerd who has to prove his l33tness on forums. It makes me feel superior to others whom I dont even know ;)
_________________

In our case php is fine due to the fact that we have to wait for other servers across the internet to actually give us a response and send us back data. We are happy with a page load of under a second.

This thread is too funny... maybe we need an off topic nerdfest.. At least we are not discussing the Linux/FreeBSD whatever distribution wars...
_________________
-Justin
--
'74 tii
'76 02
'97 m3
http://www.bmw2002.net

johnhup wrote:

See now I am going to have to crack a PHP joke ;). Java make take more overhead but performs ten times better once its running! The best I have seen PHP do is server 2-3 pages a second without caching! That in unacceptable in a production environment. I couldnt help myself. Im a nerd who has to prove his l33tness on forums. It makes me feel superior to others whom I dont even know ;)

Java nerds are pretty impreesive I gotta say. I'm just an old perl hacker for the most part. Php syntax is similar enough that I can usually figure out whats going on.

FreeBSD/Darwin/OS X are the best operating systems evar!
_________________

johnhup wrote:

FreeBSD/Darwin/OS X are the best operating systems evar!

QNX!!

I forgot to mention, about a year ago I lab tested my application (mySimon.com), and on a RedHat EL4 64-bit box with 16gigs of RAM I was getting around 100 requests per second!
_________________

johnhup wrote:

I forgot to mention, about a year ago I lab tested my application (mySimon.com), and on a RedHat EL4 64-bit box with 16gigs of RAM I was getting around 100 requests per second!

I gotta say that mysimon.com loads pretty fast and is very usable even on my crappy assed modem connecting at 21.6 kbps!!

Nice job.. never used it before.