If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. |
|
|
Thread Tools | Display Modes |
#1
|
|||
|
|||
Running a large Java program
System specifics:
AMD x86-64 128 GB of physical memory Red Hat Enterprise Linux 3 - 64-bit kernel 2.4.21-47.ELsmp x86_64 8 cpus I am trying to run a Java program that will take a lot of memory. In one instance I give the system 90GB of heap space with: java -Xmx90g JavaProgam The program runs fine until top reports that it is using approximately 43GB and then it starts swapping. top reported about 62 - 63 GB total used on the system (not sure what else is using the memory), but I still have another 64GB. Why is the system swapping when I still have memory? Do I need to do something to configure the system to use the whole 128GB? Can one process use most of the 128GB? Thanks. |
#2
|
|||
|
|||
Running a large Java program
|
#3
|
|||
|
|||
Running a large Java program
Should I post the output of the meminfo when it is starting to swap or
would any point in time be O.K.? It takes a while to get to the point of swapping. Thanks, Marc On Jun 22, 9:45 pm, (Scott Lurndal) wrote: writes: System specifics: AMD x86-64 128 GB of physical memory Red Hat Enterprise Linux 3 - 64-bit kernel 2.4.21-47.ELsmp x86_64 8 cpus I am trying to run a Java program that will take a lot of memory. In one instance I give the system 90GB of heap space with: java -Xmx90g JavaProgam The program runs fine until top reports that it is using approximately 43GB and then it starts swapping. top reported about 62 - 63 GB total used on the system (not sure what else is using the memory), but I still have another 64GB. Why is the system swapping when I still have memory? Do I need to do something to configure the system to use the whole 128GB? Can one process use most of the 128GB? Thanks. Post the results of $ cat /proc/meminfo scott |
#4
|
|||
|
|||
Running a large Java program
On Jun 23, 12:53 pm, wrote:
Should I post the output of the meminfo when it is starting to swap or would any point in time be O.K.? It takes a while to get to the point of swapping. Thanks, Marc On Jun 22, 9:45 pm, (Scott Lurndal) wrote: writes: System specifics: AMD x86-64 128 GB of physical memory Red Hat Enterprise Linux 3 - 64-bit kernel 2.4.21-47.ELsmp x86_64 8 cpus I am trying to run a Java program that will take a lot of memory. In one instance I give the system 90GB of heap space with: java -Xmx90g JavaProgam The program runs fine until top reports that it is using approximately 43GB and then it starts swapping. top reported about 62 - 63 GB total used on the system (not sure what else is using the memory), but I still have another 64GB. Why is the system swapping when I still have memory? Do I need to do something to configure the system to use the whole 128GB? Can one process use most of the 128GB? Thanks. Post the results of $ cat /proc/meminfo scott Here is the output of another run at about 20GB of memory used by the process and other processes using memory and the system is swapping. I freed other memory and the program continued, but if I hadn't I've seen cases where the program halts with no message or other times it uses all the swap and the system is unusable. -bash-3.00$ cat /proc/meminfo total: used: free: shared: buffers: cached: Mem: 1333383839744 66657845248 66725994496 0 174751744 21924777984 Swap: 4188856768 4186075136 581632 MemTotal: 130257656 kB MemFree: 65162104 kB MemShared: 0 kB Buffers: 170656 kB Cached: 21410780 kB SwapCached: 136 kB Active: 42317496 kB ActiveAnon: 40921760 kB ActiveCache: 1395736 kB Inact_dirty: 7807336 kB Inact_laundry: 7362508 kB Inact_clean: 5877816 kB Inact_target: 12673028 kB HighTotal: 0 kB HighFree: 0 kB LowTotal: 130257656 kB LowFree: 65162104 kB SwapTotal: 4088532 kB SwapFree: 586 kB CommitLimit: 69217360 kB Committed_AS: 57812004 kB HugePages_Total: 0 HugePages_Free: 0 Hugepagesize: 2048 kB Marc |
#5
|
|||
|
|||
Running a large Java program
|
#6
|
|||
|
|||
Running a large Java program
On Jun 25, 4:05 pm, (Scott Lurndal) wrote:
writes: On Jun 23, 12:53 pm, wrote: Should I post the output of the meminfo when it is starting to swap or would any point in time be O.K.? It takes a while to get to the point of swapping. Thanks, Marc On Jun 22, 9:45 pm, (Scott Lurndal) wrote: writes: System specifics: AMD x86-64 128 GB of physical memory Red Hat Enterprise Linux 3 - 64-bit kernel 2.4.21-47.ELsmp x86_64 8 cpus I am trying to run a Java program that will take a lot of memory. In one instance I give the system 90GB of heap space with: java -Xmx90g JavaProgam The program runs fine until top reports that it is using approximately 43GB and then it starts swapping. top reported about 62 - 63 GB total used on the system (not sure what else is using the memory), but I still have another 64GB. Why is the system swapping when I still have memory? Do I need to do something to configure the system to use the whole 128GB? Can one process use most of the 128GB? Thanks. Post the results of $ cat /proc/meminfo scott Here is the output of another run at about 20GB of memory used by the process and other processes using memory and the system is swapping. I freed other memory and the program continued, but if I hadn't I've seen cases where the program halts with no message or other times it uses all the swap and the system is unusable. -bash-3.00$ cat /proc/meminfo total: used: free: shared: buffers: cached: Mem: 1333383839744 66657845248 66725994496 0 174751744 21924777984 Swap: 4188856768 4186075136 581632 MemTotal: 130257656 kB MemFree: 65162104 kB MemShared: 0 kB Buffers: 170656 kB Cached: 21410780 kB SwapCached: 136 kB Active: 42317496 kB ActiveAnon: 40921760 kB ActiveCache: 1395736 kB Inact_dirty: 7807336 kB Inact_laundry: 7362508 kB Inact_clean: 5877816 kB Inact_target: 12673028 kB HighTotal: 0 kB HighFree: 0 kB LowTotal: 130257656 kB LowFree: 65162104 kB SwapTotal: 4088532 kB SwapFree: 586 kB CommitLimit: 69217360 kB Committed_AS: 57812004 kB HugePages_Total: 0 HugePages_Free: 0 Hugepagesize: 2048 kB Marc Folks usually have swap size == memory size. Here you have only 4GB swap backing 128GB of memory. Note that at the point at which you snapshot meminfo, there is 60+GB of free memory. I'd create another swap area of say, 36GB. scott I could do that, but it really doesn't explain why I am swapping at all with 60+ GB o\0 free memory. Anyone have any ideas? Thanks, Marc |
#7
|
|||
|
|||
Running a large Java program
On Jun 25, 4:05 pm, (Scott Lurndal) wrote:
writes: On Jun 23, 12:53 pm, wrote: Should I post the output of the meminfo when it is starting to swap or would any point in time be O.K.? It takes a while to get to the point of swapping. Thanks, Marc On Jun 22, 9:45 pm, (Scott Lurndal) wrote: writes: System specifics: AMD x86-64 128 GB of physical memory Red Hat Enterprise Linux 3 - 64-bit kernel 2.4.21-47.ELsmp x86_64 8 cpus I am trying to run a Java program that will take a lot of memory. In one instance I give the system 90GB of heap space with: java -Xmx90g JavaProgam The program runs fine until top reports that it is using approximately 43GB and then it starts swapping. top reported about 62 - 63 GB total used on the system (not sure what else is using the memory), but I still have another 64GB. Why is the system swapping when I still have memory? Do I need to do something to configure the system to use the whole 128GB? Can one process use most of the 128GB? Thanks. Post the results of $ cat /proc/meminfo scott Here is the output of another run at about 20GB of memory used by the process and other processes using memory and the system is swapping. I freed other memory and the program continued, but if I hadn't I've seen cases where the program halts with no message or other times it uses all the swap and the system is unusable. -bash-3.00$ cat /proc/meminfo total: used: free: shared: buffers: cached: Mem: 1333383839744 66657845248 66725994496 0 174751744 21924777984 Swap: 4188856768 4186075136 581632 MemTotal: 130257656 kB MemFree: 65162104 kB MemShared: 0 kB Buffers: 170656 kB Cached: 21410780 kB SwapCached: 136 kB Active: 42317496 kB ActiveAnon: 40921760 kB ActiveCache: 1395736 kB Inact_dirty: 7807336 kB Inact_laundry: 7362508 kB Inact_clean: 5877816 kB Inact_target: 12673028 kB HighTotal: 0 kB HighFree: 0 kB LowTotal: 130257656 kB LowFree: 65162104 kB SwapTotal: 4088532 kB SwapFree: 586 kB CommitLimit: 69217360 kB Committed_AS: 57812004 kB HugePages_Total: 0 HugePages_Free: 0 Hugepagesize: 2048 kB Marc Folks usually have swap size == memory size. Here you have only 4GB swap backing 128GB of memory. Note that at the point at which you snapshot meminfo, there is 60+GB of free memory. I'd create another swap area of say, 36GB. scott I could do that, but it really doesn't explain why I am swapping at all with 60+ GB of free memory. Anyone have any ideas? Thanks, Marc |
#8
|
|||
|
|||
Running a large Java program
On Wed, 27 Jun 2007 13:24:32 +0000, masonmarc wrote:
AMD x86-64 128 GB of physical memory Red Hat Enterprise Linux 3 - 64-bit kernel 2.4.21-47.ELsmp x86_64 8 cpus I am trying to run a Java program that will take a lot of memory. In one instance I give the system 90GB of heap space with: java -Xmx90g JavaProgam The program runs fine until top reports that it is using approximately 43GB and then it starts swapping. top reported about 62 - 63 GB total used on the system (not sure what else is using the memory), but I still have another 64GB. Why is the system swapping when I still have memory? Do I need to do something to configure the system to use the whole 128GB? Can one process use most of the 128GB? -bash-3.00$ cat /proc/meminfo total: used: free: shared: buffers: cached: Mem: 1333383839744 66657845248 66725994496 0 174751744 21924777984 Swap: 4188856768 4186075136 581632 MemTotal: 130257656 kB MemFree: 65162104 kB MemShared: 0 kB Buffers: 170656 kB Cached: 21410780 kB SwapCached: 136 kB Active: 42317496 kB ActiveAnon: 40921760 kB ActiveCache: 1395736 kB Inact_dirty: 7807336 kB Inact_laundry: 7362508 kB Inact_clean: 5877816 kB Inact_target: 12673028 kB HighTotal: 0 kB HighFree: 0 kB LowTotal: 130257656 kB LowFree: 65162104 kB SwapTotal: 4088532 kB SwapFree: 586 kB CommitLimit: 69217360 kB Committed_AS: 57812004 kB HugePages_Total: 0 HugePages_Free: 0 Hugepagesize: 2048 kB Marc Folks usually have swap size == memory size. Here you have only 4GB swap backing 128GB of memory. Note that at the point at which you snapshot meminfo, there is 60+GB of free memory. I'd create another swap area of say, 36GB. scott I could do that, but it really doesn't explain why I am swapping at all with 60+ GB of free memory. Anyone have any ideas? I thought Linux only supported 64GB. Since I've never had more than 1 I haven't worried about it. I notice you don't have any hugepages and your hugepage size is the default 2MB. I know the hugepage size goes to at least 16MB from the little research I've done on them, but I'm still not sure exactly what they are for. Perhaps they are for using more more than 64G of ram. This is all probably worthless wag info but I thought I'd throw it out anyway. I wouldn't even think you'd need a swap file with that much ram, but if the swapping is what's slowing you down it might be time to look into a real ram drive system for the swap file if you can't figure out how to use more than 64GB internally. I'll now get lost.:-) -- Want the ultimate in free OTA SD/HDTV Recorder? http://mythtv.org http://mysettopbox.tv/knoppmyth.html Usenet alt.video.ptv.mythtv My server http://wesnewell.no-ip.com/cpu.php HD Tivo S3 compared http://wesnewell.no-ip.com/mythtivo.htm |
#9
|
|||
|
|||
Running a large Java program
Wes Newell writes:
ideas? I thought Linux only supported 64GB. Since I've never had more than 1 I 32-bit linux is limited to 64GB on ia32 due to PAE. On AMD64 or EMT64 64-bit linux is limited to 1TB by the physical bus width on the current processor generations. Next gen Opteron will support more. Hugepages are only used to reduce TLB usage for applications with large datasets. The size of a hugepage is an architecture dependency. Large pages on ia32 are 4MB. on x86_64, they are 2MB. scott |
#10
|
|||
|
|||
Running a large Java program
|
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Java | C & A Bredt | Dell Computers | 4 | July 1st 05 10:54 PM |
GeForce2 MX returns to Windows after Rollercoaster Tycoon 3 starts,program still running | Bill | Nvidia Videocards | 10 | December 27th 04 05:31 PM |
Dell Dimension 3000 drivers "Program too large to fit in memory" - Deleted mysterious FAT32 partition by mistake! | Loki | Dell Computers | 2 | September 12th 04 12:58 PM |
"Out Of Memory error when trying to start a program or while program is running" | Dharmarajan.K | General Hardware | 0 | June 11th 04 10:42 PM |
My screen zooms when I run a java program | Bora Ugurlu | Nvidia Videocards | 0 | April 1st 04 10:58 PM |