New graphics for MoO?

[Zeraan]

I haven't had any luck in getting my "encoder" to work, hence i wasn't able to test the palette and transparency stuff. So you're somewhat more experienced than I am in this field. But I'm just glad that there's someone else working on this as well, so I can collaborate with him!

Glad you found the version text. Yeah, all of this work was done to figure out how to change from 1.3 to 1.4 in that graphic file

As for support files for LBX Manager: The intent was that you export out the graphics first using LBX Manager's export button, then modify those graphics, then import it back in. This exports every frame from every file within the loaded LBX, as well as the LBXHEADER.TXT. The LBXHEADER file is for LBX Manager so it can know how many frames, any special palettes, etc to correctly form the LBX's header within the LBX file.

If you can share your encoding algorithm for compressing graphics into bytes that works with LBX, I can compare it against mine and see if I can find out what's wrong with mine.

The idea is that if you were to export the graphic files, then immediately import it back in without making any changes, the new LBX file should be identical to the old one. That's when you'd know that the LBX manager is working.

I'm at work right now, but when I wrap things up at work, I'll take another stab at this if you share the algorithm

[Zeraan]

I have some extra time (wrapped things up at work) so I decided to take another look at my code. In my original post in kyrub's patch thread, I mentioned that there's two issues:

1. Frames after first one is compressed into 0 bytes.

2. There seem to be an issue with compression algorithm. But I couldn't test this due to the first issue.

Anyway, looking through my code, and debugging it, I found the cause for the first issue! It was looping through the first column of the image repeatedly, and since they're the same, setting 0 for everything.

So I added this line in "if currentY == height" block:
//Move right one
iter += height;

This fixed the first issue! I exported then imported the images, but they apparently weren't compressed correctly. I tested INTRO.LBX, the original file is 1.8 MB, but the new one that I generated was 1.55 MB. I decided to see if it would work in MoO, so I replaced the LBX file, but it crashed

Then I got an idea, export images, then import them (generating the new LBX), then try and load the new LBX in my tool. I tried to do this, and got "index out of bounds" error, so there is definitely something wrong with my compression process. I'm still looking into this, but at least I'm making progress!

Edit: I found one problem, the file sizes weren't being set correctly. That's fixed, so the tool don't give me out of bounds error anymore. But the graphics are all black... I'm done for now, maybe you can take a look at the compression algorthim and see if you can spot something? It should be the reverse of the image loading process in theory. Here's the latest LBX Manager version that has those two issues fixed: http://www.mediafire.com/?atga26fa5h03ozp

[Vanshilar]

I'll take a look at your updated code in a bit. Yeah I'll try export then import and see what happens.

In the meantime though, here's my code for the encoder. There are some notes on it though:
1. It only uses the header byte 0 method of encoding (and doesn't use the header byte 128 dec or 80 hex method of encoding). This isn't a big deal for ships (which predominantly use header byte 0) but is a big deal for larger images such as the title image, which uses the header byte 128 dec/80 hex method to save hard drive space. My original plan was to eventually have the encoder code each column both ways and then use the method that takes up less bytes.
2. It's highly simplistic in the header info and doesn't take into account all the specific stuff like custom palettes.
3. The intended input format is given in a previous post (the Slylandro Probe image). The assumed format is .png or .bmp or any other format supported by Matlab's imread function (because Matlab is very high level, the built-in imread function automatically handles image file format recognition and converting it to 8-bit RGB values). The image is 5 frames of 32 x 24 with a single line of whitespace in between each, for a total dimension of 32 x 124.
4. The palette is assumed to be the palette in FONTS.LBX; custom palettes are not supported. The code basically just does a lookup function for each pixel.

Code:

function [ lbx_output ] = LBXShipEncode( input_file, palette )
% This function takes an image file and converts it into the LBX data
% format used for ship sprites in Master of Orion
% The format for the image file should be 5 frames of 32 x 24 pixels, laid
% out vertically (first frame on top), with a 1 x 24 "white" line in
% between them to separate them

% This part checks that the input file is the right size
RGB_data = imread(input_file);
if all(size(RGB_data) == [124 32 3]) ~= 1
    disp('ERROR: input file must be 124 x 32 pixels, terminating');
    return
end

image_width = 32;
image_height = 24;

% This part converts the image file into MoO LBX palette indices
% Currently the RGB values have to be exact, so the conversion needs to be
% done prior to this; future version should have it convert automatically
% by looking for the closest match
RGB_converted = uint8(zeros(size(RGB_data,1),size(RGB_data,2))); % this is the image in palette indices
for i = 1:size(RGB_data,1)
    for j = 1:size(RGB_data,2)
        % this matches the RGB value for a given pixel to one in the
        % palette. This can probably be cleaned up.
        RGB_to_match = reshape(repmat(RGB_data(i,j,:),size(palette,1),1),size(palette,1),3);
        RGB_match = all((palette(:,1:3) == RGB_to_match),2);
        if sum(RGB_match) == 0
            disp('ERROR: no palette match found');
            [i j]
        end
        RGB_converted(i,j) = find(RGB_match,1)-1;
    end
end

% This part converts the palette data into the lbx format
% For now, it will only use the 0-encode format (regular) format, where the
% data is # transparent pixels followed by pixel data, and does not use the
% 128-encode format, where the data is # repeats followed by pixel color
lbx_output = uint8(zeros(5000,1));

% This is the header of the ship subfile. It gives the dimension of the
% image, the number of frames (5), and the byte for the first frame (42)
lbx_output(1:22) = [32;0;24;0;0;0;5;0;0;0;0;0;0;0;0;0;0;0;42;0;0;0];

% This codes frame 1
cbyte = 43; % note this is using matlab's index (starts at 1); the file index starts at 0; cbyte = "current byte"
lbx_output(cbyte) = 1; % frame 1 always starts off with a '1', unknown what it means
cbyte = cbyte + 1;
for ccolumn = 1:image_width % do the following for each column; ccolumn = "current column"
    crow = 1; % crow = "current row"
    if sum(RGB_converted(1:image_height,ccolumn)) == 0 % if column is all 0's, then just a 255
        lbx_output(cbyte) = 255;
        cbyte = cbyte + 1;
    else
        lbx_output(cbyte) = 0;
        cbyte = cbyte + 1;
        byte_start = cbyte; % this marks the byte that says how many bytes of data in this column
        cbyte = cbyte + 1;
        while crow <= image_height
            pixel_first = find(RGB_converted(crow:image_height,ccolumn) ~= 0,1) + crow - 1; % the first non-zero pixel
            if isempty(pixel_first) % this means that all the remaining pixels are 0
                % then no further bytes of encoding are needed, so while
                % loop can terminate; assumes that the code has already
                % checked for all-0 column, above
                break
            end
            if pixel_first == image_height
                pixel_last = image_height;
            else
                pixel_last = find(RGB_converted(pixel_first+1:image_height,ccolumn) == 0,1) + pixel_first-1; % the last non-zero pixel of the current sequence
                if isempty(pixel_last) % this means the pixels extend to the end of the column
                    pixel_last = image_height;
                end
            end
            lbx_output(cbyte) = pixel_last - pixel_first + 1;
            lbx_output(cbyte+1) = pixel_first - crow;
            lbx_output(cbyte + 2 + (0:(pixel_last - pixel_first))) = RGB_converted(pixel_first:pixel_last,ccolumn);
            cbyte = cbyte + 2 + pixel_last - pixel_first + 1;
            crow = pixel_last + 1;
        end
        lbx_output(byte_start) = cbyte - byte_start - 1;
    end
end

% This codes frames 2-5
for frame = 2:5
    lbx_output(cbyte) = 0; % frames 2-5 always starts off with a '0', unknown what it means
    % frame 2 is byte 23-24, frame 3 is byte 27-28
    lbx_output(15+4*frame) = mod(cbyte-1,256);
    lbx_output(16+4*frame) = floor((cbyte-1)/256);
    cbyte = cbyte + 1;
    image_change = RGB_converted((frame-1)*(image_height+1)+(1:image_height),:) ~= RGB_converted((frame-2)*(image_height+1)+(1:image_height),:);
    for ccolumn = 1:image_width % do the following for each column; ccolumn = "current column"
        crow = 1; % crow = "current row"
        if sum(image_change(:,ccolumn)) == 0 % if column is all 0's, then just a 255
            lbx_output(cbyte) = 255;
            cbyte = cbyte + 1;
        else
            lbx_output(cbyte) = 0;
            cbyte = cbyte + 1;
            byte_start = cbyte; % this marks the byte that says how many bytes of data in this column
            cbyte = cbyte + 1;
            while crow <= image_height
                pixel_first = find(image_change(crow:image_height,ccolumn) ~= 0,1) + crow - 1; % the first changed pixel
                if isempty(pixel_first) % this means that all the remaining pixels are 0
                    % then no further bytes of encoding are needed, so while
                    % loop can terminate; assumes that the code has already
                    % checked for all-0 column, above
                    break
                end
                if pixel_first == image_height
                    pixel_last = image_height;
                else
                    pixel_last = find(image_change(pixel_first+1:image_height,ccolumn) == 0,1) + pixel_first-1; % the last non-zero pixel of the current sequence
                    if isempty(pixel_last) % this means the pixels extend to the end of the column
                        pixel_last = image_height;
                    end
                end
                lbx_output(cbyte) = pixel_last - pixel_first + 1;
                lbx_output(cbyte+1) = pixel_first - crow;
                lbx_output(cbyte + 2 + (0:(pixel_last - pixel_first))) = RGB_converted((frame-1)*(image_height+1)+(pixel_first:pixel_last),ccolumn);
                cbyte = cbyte + 2 + pixel_last - pixel_first + 1;
                crow = pixel_last + 1;
            end
            lbx_output(byte_start) = cbyte - byte_start - 1;
        end
    end
end

lbx_output(39) = mod((cbyte-1),256);
lbx_output(40) = floor((cbyte-1)/256);
lbx_output = lbx_output(1:(cbyte-1));

end

In Matlab, the "palette" input is simply a matrix of the RGB values for each entry in uint8 format:

Code:

0    0    0
0    0    0
28    28    28
44    44    44
60    60    60
77    77    77
89    89    89
121    121    121
134    134    134
150    150    150
166    166    166
195    195    195
211    211    211
227    227    227
239    239    239
182    182    207
158    158    190
138    138    174
117    117    158
101    101    142
85    85    125
69    69    109
56    56    93
40    40    77
28    28    60
20    20    48
12    12    32
24    12    0
36    20    0
60    32    0
101    48    0
113    56    0
138    69    0
154    65    0
134    44    0
121    32    0
109    24    0
85    12    0
73    4    0
60    8    8
44    32    24
52    40    32
65    48    48
69    52    40
69    69    44
52    56    20
36    44    16
36    28    20
97    12    12
12    20    8
20    16    12
40    4    4
56    28    20
69    32    24
77    40    28
97    40    40
105    52    40
130    69    48
97    77    60
93    89    60
93    73    73
85    52    65
81    65    52
117    16    16
138    20    20
158    24    24
178    28    28
199    32    32
215    32    32
235    36    36
255    40    40
215    138    0
231    166    0
255    142    0
203    121    0
190    105    0
178    93    0
166    77    0
146    117    0
142    105    28
166    121    40
186    138    56
211    154    69
231    174    81
255    190    56
255    223    81
227    227    40
195    150    121
154    134    125
146    117    117
146    113    93
130    101    81
117    109    73
113    89    73
117    93    93
150    97    73
101    16    8
162    125    101
178    138    113
170    158    130
199    182    154
121    150    158
73    130    166
65    113    146
52    93    121
44    77    101
16    52    97
12    40    77
20    40    12
32    77    16
36    93    20
44    113    20
52    150    28
73    207    36
77    142    105
44    138    138
28    109    109
16    77    77
36    93    65
56    117    85
65    130    97
142    130    89
73    36    73
65    0    65
93    0    81
48    24    16
178    93    69
178    113    89
48    0    4
97    0    12
150    0    24
203    0    32
211    36    69
223    85    109
231    138    154
243    195    203
48    8    0
97    24    0
150    36    0
203    52    0
211    85    36
223    121    85
231    162    138
243    207    195
48    24    0
97    52    0
150    81    0
203    109    0
211    134    36
223    162    85
231    190    138
243    223    195
48    32    0
97    73    0
150    113    0
203    150    0
211    166    36
223    186    85
231    207    138
243    231    195
211    113    0
231    125    0
150    150    0
203    203    0
211    211    36
223    223    85
231    231    138
243    243    195
32    44    0
69    89    0
105    138    0
142    186    0
162    199    36
182    211    81
203    227    134
227    239    190
0    36    0
0    73    0
0    113    0
0    154    0
28    174    28
73    195    73
125    215    125
186    235    186
0    40    28
0    85    60
0    130    93
0    174    125
32    190    146
77    207    170
130    223    195
186    239    223
0    0    48
0    0    97
0    0    150
0    0    203
36    36    211
85    85    223
138    138    231
195    195    243
20    0    48
48    0    97
73    0    150
97    0    203
121    36    211
154    85    223
182    138    231
219    195    243
36    0    36
77    0    77
117    0    117
158    0    158
174    32    174
195    73    195
215    125    215
235    186    235
48    0    20
97    0    48
150    0    73
203    0    97
211    36    121
223    85    154
231    138    182
243    195    219
32    20    8
73    44    20
113    73    32
150    97    48
170    121    77
190    150    113
211    182    154
231    215    203
8    20    32
20    44    73
32    73    113
48    97    150
77    121    170
113    150    190
154    182    211
203    215    231
20    20    0
48    48    0
73    73    0
97    97    0
125    125    20
158    158    60
190    190    113
223    223    174
0    0    0
32    32    32
69    69    69
105    105    105
142    142    142
178    178    178
219    219    219
255    255    255

If you happen to have Matlab, you can run this directly from the directory that the image file is in. The output is the data for the ship image in LBX format (i.e. that I posted in a previous post).

The pseudocode goes something like this:

Code:

read and convert image file to RGB values
convert RGB values to palette index values by doing the following:
    for each row
        for each column
            find the first entry in the palette table that matches this pixel's RGB value
            RGB_converted's corresponding pixel location's value is the palette index where the match was found
(at this point, the image has been converted to the palette values for LBX use, so the rest is what's more important)
input some header data for the LBX output
for the first frame:
    current byte ("cbyte") = 43 (43 in Matlab is 42 in the file since Matlab's indexing starts at 1, not 0)
    lbx_output(cbyte) = 1 (frame 1 always starts off with 1 while other frames start with 0, not sure what it's for)
    cbyte = cbyte + 1 (advances cbyte by 1)
    for each column
        current row ("crow") = 1 (starts off at the first row)
        if all values in RGB_converted's column is 0
            then input a 255 to lbx_output and advance current byte by 1
        else
            input a 0 to lbx_output and advance current byte by 1 (encoding format 0, encoding format 128 not yet supported)
            mark location of cbyte (will be filled in later with total length of bytes)
            while crow hasn't reached the bottom of the image yet
                find the location of the first non-zero pixel between crow and the bottom of the column
                if it's not found, assume the remaining pixels are all zero --> break out of while loop
                if it's the last pixel in the column, then it's also the last pixel
                else find the last non-zero pixel in the column
                input number of non-zero pixels to lbx_output and advance current byte by 1
                input number of zero pixels to skip to lbx_output and advance current byte by 1
                input palette values for non-zero pixels to lbx_output and advance current byte by that many values
                advance crow to next row after the already-encoded rows
            once the while loop ends, add the total number of bytes added (from the change to current byte) and input that value to the previously marked location

The pseudocode for frames 2 to 5 are pretty similar to this (in fact, I copied over the code from the one for frame 1 and then modified it as needed). The main differences are:
1. It inputs the frame location bytes in the header
2. There is an "image_change" matrix which is whether the current frame changed from the previous frame (0 = no change and 1 = change) for each pixel. All the logic is then based off of this frame, with the exception of rather than inputting all the 1's into lbx_output, it inputs the palette values of the current frame at their location.

I'll look into converting this matlab script to a .exe file, I think there are ways to do that.

[Vanshilar]

Here are some notes on the LBXManager compression for images. I tested it on SHIPS.LBX and compared the new with the original file:

1. It currently starts the data at byte 39. It should be byte 42 however because bytes 38-41 are reserved for the last frame location (i.e. the hypothetical frame 6 which marks the end of frame 5).
2. It doesn't seem to take into account blank columns correctly (especially for repeated frames). I think it's because the while loop that leads to 0xFF requires "frame[iter + i] == 0x80"; I'm not sure what the purpose of this is.
3. For the rest of it, I'm still looking through the code, I'm not quite sure yet how it's trying to encode things. For SHIPS.LBX, the data in the first entry (the first RSMALL) should be a lot of FF followed by "00 04 02 0B 81 81" to encode the 9th (starting from 1) column (which is just 2 pixels of palette 129 at rows 12 and 13 (starting from 1), but the LBXManager is starting off with "00 05 03 0C FA FB FA" which I'm not sure what they're for since I don't see that row in the image. It might be useful to output some intermediate variables (for example, the palette entries that it's converting, i.e. "00 00 00 00 00 00 00 00 00 00 00 81 81 00 00 00 00 00 00 00 00 00 00 00" for the 9th column of the first RSMALL in SHIPS.LBX, which should end up as "00 04 02 0B 81 81" or "80 04 02 0B E1 81" depending on the encoding method) to see if they're matching up to what's expected.

[Zeraan]

Alright, thanks for your observations! I'm not sure when I'll have a chance to work on it again. When you mentioned palette lookup, I realized that I'm not doing that during the compression, so I think that may be one of the problems with the compression. I'll have to make sure it's using a palette...

[Vanshilar]

I've coded up 6 of the Star Control 2 ships into MoO's LBX format. The longest part of the process is actually re-drawing the SC2 ships and then matching them with MoO's palette. The ships are the Ariloulaleelay Skiff, the Thraddash Torch, the Slylandro Probe, the Ur-Quan Dreadnought, the Chenjesu Broodhome, and the Chmmr Avatar. They are shown below from a video grab from DosBox of MoO (I had to cut the right part out to make the animated gif fit Realms Beyond's 500 kb size limit):

   

I chose these ships to do first for the large part because they are more "interesting" visually, although the Skiff was more just because it's easy. Most of the ships won't be as fancy as these though once I complete the entire set of 24.

SC2's ships go up to about 38 pixels long and 37 pixels wide, while MoO's format is at most 32 pixels long and 24 pixels wide. So for most of the ships, I have to resize them by around 60-70% or so to make them fit within MoO's sizes. SC2 does have different zooms but the next zoom results in ships of up to about 19 pixels by 19 pixels, which end up being too small.

After the ships are rescaled, I then change them over to MoO's color palette. MoO's default palette doesn't span the color space as much as SC2's; for example, the Ur-Quan green in Star Control 2 is true green or (0 255 0), whereas the closest thing in MoO is something like (73 207 36). So the Ur-Quan Dreadnought in MoO ends up looking a bit muted or washed-out compared with the original in SC2, unless I want to dabble in making custom palettes. Converting the ships to MoO's palette is done manually pixel by pixel based on what I judge to be the colors that best preserve the original look.

The images are then encoded into LBX format via script, and I then copy-paste them manually into SHIPS.LBX. I then adjust the header so that they point to the right locations.

The ships that I modified are all yellow, so you'll have to play under the yellow flag to use them. I modified the 1st small, 1st medium, 1st large, and 4th, 5th, and 6th huge ships.

The modified SHIPS.LBX file is below. To use, just unzip it into your MoO directory (be sure to move the original SHIPS.LBX somewhere else for safekeeping first). There shouldn't be any problems, but let me know if you have any troubles with it.

[Arne]

That's cool! I didn't know the frames included the entire ship. I thought the exhaust was separate.

I drew a random bug ship just to see if the palette allowed for some color temperature.

[thorn]

Would it be possible for one of you to post the new working version of LBX Manager here? I want to attempt a Star Trek mod for MOO2 but if I can't change the ship graphics it's not worth doing. Thanks!

[Zeraan]

The LBX Manager that I'm working on works with MoO 1, not Moo 2. Someone has already created an utility for editing LBX files for MoO 2, it can be found here:

http://www.spheriumnorth.com/orion-forum....php?t=413

[thorn]

The LBX Manager that I'm working on works with MoO 1, not Moo 2. Someone has already created an utility for editing LBX files for MoO 2, it can be found here:

http://www.spheriumnorth.com/orion-forum....php?t=413

Thanks for the reply! Yeah, I found that utility package but it only extracts the ship graphics with no way to import them back in. I also found a picture editor that went with it but it doesn't seem to really do anything except show black squares that you can't edit.
I might try to edit the file manually.