Efficiency – Virtual Dub

Today I was reading the Blog at VirtualDub.org.
It reinforced my believe that there is too much inefficient code/shit out there in the world.
And it’s not just SW, it’s HW too. I’m horrified when I read some video card specs. with 500W power consumption… seriously who needs an oven/central heating in their computer? Talk about global warming…

There is serious problem with idle handling of the PC platform, OS support, programmers practices.
Please stop with the background tasks
The main problem is Disk IO handling, and maybe memory bandwidth, network bandwidth. Specifically the lack of priority, or rather in general that only CPU time can be prioritized. (Net QoS is not what I’m talking about)
CPU is rarely the bottleneck, when it comes to real time behavior. (video capture, GUI response)
When an “idle” task starts to read/write data from/to the disk, it can have a serious performance impact. (So does windows update background intelligent transfer on on-line games/videos…) The problem is not just large amount of data, but just disk seek (time) is enough to do some damage (dropped frames…). Direct disk usage is not even the most common cause of performance problems, but rather indirect access due to paging. When an “idle” task requests some of it’s memory that are paged out to disk, windows is happy to read them back from the page file. And memory paging is a high priority task! This is how your idle task becomes high priority… and it’s not fun when you have 1-2 GB size pagefile.

interesting/funny quotes:
About Visula Studio Intellissense/customer support:
“I’m now of the opinion that running tasks in the background is a bad idea and contributes to the opinion that computers are slow and unpredictable. In this case, the Visual Studio programmers got the idea that Intellisense could be made low impact and “invisible” by running it as a low priority background thread, which ignores the problem that it still drains a huge amount of I/O bandwidth on the hard disk. This isn’t the first time the Visual Studio team has done something like this — I love it when the .NET Framework updates and suddenly a lot of CPU is taken up by “background” NGEN tasks for minutes at a time. If everyone did this, computers would be stuck at 100% CPU for hours at a time with no apparent reason.”
“We’ve now gotten to the point where computers are fast enough to perform many everyday computing tasks much faster than necessary, which unfortunately has given rise to the opinion that we don’t need efficient code anymore. While it’s true that there’s no need to make a window come up in 0.5ms when it currently comes up in 5ms, I think the recent trend toward low-power, quieter, and lighter-weight devices is providing a new reason for efficient code. For many programs, it’ll no longer be about making the program run faster, but to make it run more efficiently, letting the CPU and GPU idle more, allowing the hard drive and fans to spin down, and prolonging battery life on laptops. This is nothing new to people who worked on older platforms or on mobile devices, but this kind of thinking isn’t as common in the desktop world.”
“I should note that it’s not actually that easy, because Intellisense is expected to return useful data from files that don’t even come close to compiling, whereas the compiler is expected not to spew 2MB of errors because you forgot a semicolon.”
“”we can’t fix this horrible bug, because then we couldn’t fulfill this design requirement”–or put differently, when developers refuse to acknowledge something like “doesn’t thrash the disk for minutes at a time” as a requirement, because they’d have to admit that they’re not capable of fulfilling all of the requirements. Of course, this leads to many of those absurd “this behavior is by design” KB entries…”
“Since performance bugs are dependent upon workload and machine configuration, you can’t gauge a performance bug in binary terms unless you are running specific test cases or have an egregiously bad case, like an infinite loop or close enough (projected running time exceeds the lifetime of the universe).

by-the-by Borland Developer Studio 2006 is at least as bad as Visual Studio. I used and still use Borland Delphi 7 but you cannot buy it anymore, so I bought BDS 2006. It’s full of bugs, performance hogs…

Dymanic Disk – Fault Tolerant Volumes on XP (SP3)

This is a great article: Using Windows XP to Make RAID 5 Happen
Here is the important part: Windows XP has the ability to create/handle Mirrored, RAID 5 volumes, but it is “disabled”. There is a way to “enable” these features. This hack is described in the mentioned article.

In case it’s not available there:
you need to modify these 3 files:
C:\windows\system32\drivers\dmboot.sys

Before:
0000f1f0h: 74 00 54 00 79 00 70 00 65 00 00 00 57 49 4E 4E ; t.T.y.p.e…WINN
0000f200h: 54 00 00 00 53 45 52 56 45 52 4E 54 00 00 00 00 ; T…SERVERNT….
After:
0000f1f0h: 74 00 54 00 79 00 70 00 65 00 00 00 53 45 52 56 ; t.T.y.p.e…SERV
0000f200h: 45 52 4E 54 57 49 4E 4E 54 00 00 00 00 00 00 00 ; ERNTWINNT…..

C:\windows\system32\dmconfig.dll

Before
00005140h: 4C 41 4E 4D 41 4E 4E 54 00 00 00 00 53 45 52 56 ; LANMANNT….SERV
00005150h: 45 52 4E 54 00 00 00 00 57 49 4E 4E 54 00 00 00 ; ERNT….WINNT…
After
00005140h: 4C 41 4E 4D 41 4E 4E 54 00 00 00 00 57 49 4E 4E ; LANMANNT….WINN
00005150h: 54 00 00 00 00 00 00 00 53 45 52 56 45 52 4E 54 ; T…….SERVERNT

C:\windows\system32\Dmadmin.exe

Before
00001c30h: 73 65 72 76 65 72 6E 74 00 00 00 00 6C 61 6E 6D ; servernt….lanm
00001c40h: 61 6E 6E 74 00 00 00 00 50 72 6F 64 75 63 74 54 ; annt….ProductT
After
00001c30h: 77 69 6E 6E 74 00 00 00 00 00 00 00 6C 61 6E 6D ; winnt…….lanm
00001c40h: 61 6E 6E 74 00 00 00 00 50 72 6F 64 75 63 74 54 ; annt….ProductT

and copy a copy to C:\windows\system32\dllcache
But don’t forget about windows file protection, do this in safe mode, or form boot cd/usb…

Windows service pack 3 overwrites these files (2 of them anyway). So if you did this on SP2 and updated to SP3 after that you won’t be able to access mirrors, raid 5… The good news is this works on SP3 too, so all you need to do is reapply this “patch”. (A new windows update, or “security patch” might disable this feature in the future. But I think microsoft is concentrating on Vista these days)

Why is this good/useful anyway?
First of all, do you want to keep your data safe?
Let’s be clear, this alone won’t keep your data safe, this is just one step. You need external backup (possibly a location far away) too.
There is several reason for and against hardware raid. My top reasons against HW and for SW RAID is: HW change/compatibility/portability, completely custom raid volume sizes/disk sizes. (but RAID 5 is slow on SW raid)
Well if you do not have windows 2003, but you want to keep your data on SW RAID, this might be your solution.
Personally I do have a win2k3, but only one. And if it’s not working for some reason I want to be able to get to my data fast from a second computer (win XP running on it). I don’t use raid on my backup xp, but I might need it someday…

Dynamic Disk Group Name vs. ID

Today I had another encounter with dynamic disks. I recently removed 4 disk from my computer and replaced them. Now I have 4 spare disk and I plan to use them for backup purposes. I put the first disk in a PC (running XP) and started the Disk management console. I imported the foreign disk, everything went well… I removed the first and put in the second disk, this time I did not have to import it. I did not need to import the third one either. Than I put in the 4th, but now I cannot import this dynamic disk. In the Event Viewer it says:
LDM – INTERNAL Error – A disk group with the specified name already exists (C100007A).

The next part is For Experts only! Backup your data! You’ve been warned!
Lets start with the tools:
You can find some useful utils on the windows install cd (who would have thought :)) \support\tools\suptools.msi

To generate detailed information on your dynamic disk configuration run:
\Program Files\Support Tools\dmdiag.exe -v >> disk.log

Here is parts of the result for disk #3:

———- Dynamic Disk Information ———–
DiskGroup: FotonDg0
Group-ID: 7*******-****-****-****-************

———- LDM Disk Header Harddisk1 ———-
diskid: 1xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
group: name=FotonDg0 id=7*******-****-****-****-************
flags: private autoimport
import: bootsig=4iiiiiiii hostid=hhhhhhhh-hhhh-hhhh-hhhh-hhhhhhhhhhhh
diskset: id=6sssssss-ssss-ssss-ssss-ssssssssssss new=asssssss-ssss-ssss-ssss-ssssssssssss

———- LDM Disk Config Harddisk1 ———-
#dgname: FotonDg0 dgid: 7*******-****-****-****-************

For Disk #4:

———- Dynamic Disk Information ———–
DiskGroup: FotonDg0
Group-ID: 2*******-****-****-****-************

———- LDM Disk Header Harddisk1 ———-
diskid: axxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
group: name=FotonDg0 id=2*******-****-****-****-************
flags: private autoimport
import: bootsig=1iiiiiiiii hostid=hhhhhhhh-hhhh-hhhh-hhhh-hhhhhhhhhhhh
diskset: id=fsssssss-ssss-ssss-ssss-ssssssssssss

———- LDM Disk Config Harddisk1 ———-
#dgname: FotonDg0 dgid: 2*******-****-****-****-************

The hostid and the group name is the same for both disks.
It seems I have multiple disk group id for the same disk group name.
How this happened I don’t know, but not really the question either. The Question is how to fix it.

First of all, at the moment this is the only dynamic disk connected to the PC, so it shouldn’t be a problem importing it (except it is). The problem is that windows stored the first diskgroup id in somewhere the registry. I suspect HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\dmio\Boot Info\Primary Disk Group. Since I have the ID at hand (saved dmdiag log) lets search for it. …searching… Deleting this key, (or modifying to the 4th disk’s group ID) should fix this problem. (reboot might be required) (search done: HKEY_LOCAL_MACHINE\SYSTEM\ControlSet*\Services\dmio\Boot Info\Primary Disk Group were the only places.)

But what if I want to use both disk (with different disk group ID) together? How can I fix the disk group ID mismatch / same name? What’s the right way? How Can I edit the disk group name and/or ID ? If I manage to rename the disk group, will windows handle multiple disk groups?
Well I don’t have the answers for these questions (yet) 🙁

In a disk group each and every disk has a copy of configuration data of all disks and volumes in that group. Modifying completely different disk groups to the same name and id is probably not a good idea, since both have completely different configuration data on them. I’m not sure how would the windows deal with the conflict. Conflict in the configuration data may occur even if the disks were in the same group at some point of time. for example if a disk is removed from the system, and then new volumes were created or removed… That’s why I imported every disk separately.

Since I plan to convert all these disks to basic disks I don’t really need to care, but it would be nice to know these things for future reference. My advice is do not separate dynamic disks from the group,(and do not rejoin them later,) move them together. (Or do not use them at all, at least not for mobile data storage)

Here is some info on Dynamic Disks
Best practices for using dynamic disks on Windows 2000-based computers
Description of Disk Groups in Windows Disk Management

Here is an other useful tool: diskpart
for example to wipe your hdd clean with diskpart:
>diskpart
select disk 1
detail disk
clean all !!! do NOT do this unless you are absolutely sure
(This does not provide complete data destruction. It might be possible to read your data with special hardware based on electromagnetic residue)

Dynamic Disk Group ID problem Solved

This Article solved my problem, although it’s for windows 2000 it works like a charm on windows 2003 too:
You receive a “Stop 0x0000007B” error message when you try to start your computer after you move the dynamic hard disk
This is the only article on support.microsoft.com that describes this problem correctly and provides a solution. This “APPLIES TO: Microsoft Windows 2000” but works on 2003 too which is good since there is no article for 2003 for this problem.
This is one of the few article that was written by experts who know what they are talking about, and contains real information on the inner workings of the system. Most of the troubleshooting manuals are just PR bullshit or written for illiterate ppl.

Just to be on the safe side here is the solution:
BSoD: STOP: 0x0000007b (0xf881b84c,0xc0000034,0x00000000,0x00000000)
Means: INACCESSIBLE_BOOT_DEVICE
0xc00000034: STATUS_OBJECT_NAME_NOT_FOUND

    Cause:

This error message may occur if the primary disk group identity of the hard disk does not match the primary disk group identity that is stored in the registry. This mismatch may occur if all the following conditions are true:
• You configure the hard disk that contains the system partition as a dynamic disk on a computer that is running Windows 2000. (or 2003)
• You remove the hard disk from the computer, and then install the hard disk in a second Windows 2000-based computer. (or 2003)
• You import the hard disk to a disk group that contains dynamic disks on the second Windows 2000-based computer. (or 2003)
• You return the hard disk from the second computer to the first computer.

In Windows 2000, there can be only one dynamic disk group. When you move a dynamic disk from one computer to a second computer that already contains dynamic disks, the primary disk group identity on the disk is changed, and the disk is merged into the second computer’s dynamic disk database. However, the primary disk group identity that is stored in the registry of the operating system on the disk is not changed. When you return the hard disk to the first computer, the mismatch between the new primary disk group identity and the primary disk group identity that is stored in the registry causes the error.

    Resolution:

To resolve this problem, use Registry Editor to delete the Primary Disk Group registry key from the registry of the computer that does not start. ([HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\dmio\Boot Info\Primary Disk Group])
Back up the registry before you modify it!
To delete this key, follow these steps:
1. Boot from an other windows.
2. Run Regedt32 / regedit
3. In Registry Editor, click HKEY_LOCAL_MACHINE, and then on the File menu, click Load Hive…
4. Locate the “corrupt” System file that contains the hive of the operating system that needs fixing (%SystemRootOfCorruptWin%\System32\Config\System.)
5. type Temp in the Key Name box, and then click OK
6. Navigate to HKEY_LOCAL_MACHINE\Temp\SYSTEM\ControlSet*\Services\dmio\Boot Info\Primary Disk Group
7. Right-click the Primary Disk Group registry key, and then click Delete.
8. (repeat 6.,7. for every ControlSet )
9. Click Temp, and then on the File menu, click Unload Hive…, and then click Yes.
10. quit regedit, shut down windows
11. Boot the fixed windows

It’s hard to find info on this code that does not just say “is usually caused by a missing or corrupt driver”.

might be useful:
How to troubleshoot the stop error 0x0000007B?
How to troubleshoot “Stop 0x0000007B” error messages that occur when you run Windows 2000 Setup
How to troubleshoot “Stop 0x0000007B” error messages in Windows 2000

Win 2008 Server Setup crap

I had an interresting day yet again 🙁
Windows 2008 Server (x86) [codename Longhorn] [win2k8] setup corrupted my existing Windows 2003 Enterprise Server!
The problem might be traced back to Dynamic Disk Group ID problem (and not related directly to the win2k8 setup at all) and might be caused by the dualboot config.
After I cancelled the win 2008 install and tried to boot my win 2003 it failed with STOP: 0x0000007B (0xF789EA98,0xc0000034,0,0) BSOD (Blue Screen of Death).
One (yet) unconfirmed explanition is Dynamic Disk Group ID mismatch between the one stored on the disk configuration information (1MB at the end of the disk) and the one stored in the registry (HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\dmio\Boot Info\Primary Disk Group). How and why this happened is unccelar at this momnet.

Here are some info on the topic:
Description of Disk Groups in Windows Disk Management
Dynamic Disk Numbering and the DmDiag.exe Tool
How to troubleshoot “Stop 0x0000007B” error messages in Windows 2000 — the best guess on solving the problem (yet)

Useful tools for (data recovery) experts:
Dmdiag
DiskProbe

Details:
I had 2 working Windows 2003 Enterprise Server on my computer (a Main and a Backup). The Main win2k3 was running on mirrored dynamic disk volumes. The Backup is (still) running on Basic Disk.

    Disk Setup:

Basic Disk on Motherboard PATA Primary Master – Backup Win2k3
DVDRW on MB PATA P. Slave
Dynamic Disk on MB PATA Secondary Maste – SW mirror (RAID1) – Main win2k3
Dynamic Disk on MB PATA S. Slave – Some SW mirrors and simple volumes – (intended win2k8 target)
Basic Disk on MB SATA (Si3112r) Ch. 0
Dynamic Disk on MB SATA (Si3112r) Ch. 1 – SW mirror – Main win2k3 mirror and others
Basic Disk on SATA Raid Card (Si3114r) Ch. 2
Basic Disk on SATA Raid Card (Si3114r) Ch. 3

    BIOS Settings:

Boot Order: CDROM, HDD-1, SCSI
(HDD-1 is MB S. Master – Main win, HDD-0 is backup win)
Main win was working perfectly (for years), backup win was installed 15 days ago on a single disk.

    The Process:

Boot from Win 2008 Install DVD, choose language, exit / cancell setup. On the install screen i closed the window. I wanted to do a backup first. As it turns out I was already late 🙁 The setup said the computer “might restart” if i cancel the install, and it did. I left the DVD in the drive, but did not boot from it rather let it boot to HDD. but instead of booting the hdd-1 it booted to the backup win (hdd-0). I tried to chane bios boot order with no effect before I realised the DVD was booting the hdd-0 not the bios. So I removed the DVD and restored the original boot sequence. (I think i mught tried to boot the main win form the backup win boot menu (hdd-0/boot.ini)) Although my main win boot menu was displayed it did not boot correctly. After displaying the progress bar the computer restarted. I disabled auto reboot and ther it was our favourite BSOD with the message:
STOP: 0x0000007B (0xF789EA98,0xc0000034,0x00000000,0x00000000)

I booted my backup win and googled the hell out of it, and came up with a bunch of misssing /corrupted driver explanation, witch seens highly unlikely, since it was just working fine a minute ago, and nothing was changed on the HW (nor should have on the SW). I found a very similar case here:
http://social.technet.microsoft.com/forums/en-US/winserversetup/thread/19402d17-3239-4e8b-b6c6-e06f45f3a123/
His config:
Disk 1 active partition/volume boots to Windows 2000 Professional
Disk 2 active partition boots to Windows Server 2008 (setup failed)
Disks 3/4 contains mirrored volumes. Active partition/volume boots to Windows Server 2003
“This system was working fine until I attempted to install Windows Server 2008 onto Disk 2”
http://support.microsoft.com/kb/314082/

SMTP 4 – mail relay

Vegre volt egy kis idom osszedobni az E-Mail Relay programomat ami egy e-mail forwarder/tovabito. A program egyszerre e-mail szerver es kliens egyben. A szerver resze fogadja a leveleket, lementi fileokba, majd tovabbkuldi egy masik SMTP szervernek mintha egy e-mail kliens lenne. Hogy miert is jo ez? Azert, mert a szerver resz nem ker azonositast (viszont csak a localhostrol fogad leveleket). Igy egyreszt nem kell a levelezo kliensekben minden accounthoz bealllitani a felhasznalonevet, jelszot (jelszovaltoztataskor nem kell mindenhol atirni). Masreszt (ami miatt az egeszet csinaltam), a php mail() fuggvenye nem tamogatja sem az authentikaciot, sem a titkositast. Az en programom viszont igen. Igy a szerver altal kapott leveleket a program kliens resze feldolgozza: bejelentkezik a megfeleo felhasznalonev jelszo parossal a kivalasztott SMTP szerverre es SSL titkositott csatornan tovabbitja a leveleket.
Azert van szukseg erre a bonyolult jatekra, mert ha asajat SMTP szerveremrol kuldom a leveleket, akkor azt szinte senki sem fogja megkapni, mert a szolgaltatok kiszurik azokat. Tehat szukseg van egy kulso, megbizhato szoldaltatora, ami viszont felhasznaloi bejelentkezset kovetel meg.
Mivel nem volt kedvem atirni a php forrasat, es ujraforditani azt, hogy a mail fuggveny mukodjon a kulso SMTP-vel. Sem php-ben nem tudtam megirni sajat titkositott levelkuldest es bejelentkezest, igy jutottam el ehhez a megoldashoz.
Majd ha lesz meg idom akkor kicsinositom a programot es kozzeteszem…