MPI Question

[Locutus4657]

I am trying to send the following structure using MPI_Type_struct:

struct individual
{
unsigned *chrom;
double fitness;
int xsite;
int parent[2];
int *utility;
};

My program crashes durring send, what happens is chrom transmits successfully, but everything else seems to be messed up. I did deep copies of everything too make sure that my unsigned * and int * are equivilant to unsigned[] and int[]. I know that the offset is supposed to be of type MPI_AINT, but I havn't seen any good description of how to use something of this type. If anyone in the forum is experianced with MPI and can help that would be great. I have my program working where I send each member of each individual I am transmitting individaully, but as you can imagin this is quite inefficiant.

derive my MPI type is cut and pasted below:

floatsize = sizeof(float);
intsize = sizeof(int);
blen[0] = chromsize; byteOffSet[0] = 0; oldtypes[0] = MPI_UNSIGNED;
blen[1] = 1; byteOffSet[1] = nbytes; oldtypes[1] = MPI_DOUBLE;
blen[2] = 1; byteOffSet[2] = (nbytes + floatsize); oldtypes[2] = MPI_INT;
blen[3] = 2; byteOffSet[3] = (nbytes + floatsize + intsize); oldtypes[3] = MPI_INT;
blen[4] = ulen; byteOffSet[4] = (nbytes + floatsize + (intsize * 3)); oldtypes[4] = MPI_INT;
blen[5] = 1; byteOffSet[5] = sizeof(struct individual); oldtypes[5] = MPI_UB;

MPI_Type_struct(6, blen, byteOffSet, oldtypes, &MPI_INDI);

Here is where I'm doing my sending and reciving:
if (workstation_id < other)
{
MPI_Send(my_transmition, num_transmit, MPI_INDI, other,
num_transmit, MPI_COMM_WORLD);

MPI_Recv(temp, num_transmit, MPI_INDI, other, num_transmit,
MPI_COMM_WORLD, &status);
_heapchk();
}
else
{
MPI_Recv(temp, num_transmit, MPI_INDI, other, num_transmit,
MPI_COMM_WORLD, &status);

MPI_Send(my_transmition, num_transmit, MPI_INDI, other,
num_transmit, MPI_COMM_WORLD);
_heapchk();
}
_heapchk() is a microsoft function which verifies that the heap is in
good condition. Thank a lot.

Carlo Razzeto

 

[Armitage]

<< I am trying to send the following structure using MPI_Type_struct:

struct individual
{
unsigned *chrom;
double fitness;
int xsite;
int parent[2];
int *utility;
}; >>



mmmm ... looks like fun with GAs?



<< My program crashes durring send, what happens is chrom transmits successfully, but everything else seems to be messed up. I did deep copies of everything too make sure that my unsigned * and int * are equivilant to unsigned[] and int[]. I know that the offset is supposed to be of type MPI_AINT, but I havn't seen any good description of how to use something of this type. If anyone in the forum is experianced with MPI and can help that would be great. I have my program working where I send each member of each individual I am transmitting individaully, but as you can imagin this is quite inefficiant.

derive my MPI type is cut and pasted below:

floatsize = sizeof(float);
intsize = sizeof(int);
blen[0] = chromsize; byteOffSet[0] = 0; oldtypes[0] = MPI_UNSIGNED;
blen[1] = 1; byteOffSet[1] = nbytes; oldtypes[1] = MPI_DOUBLE;
blen[2] = 1; byteOffSet[2] = (nbytes + floatsize); oldtypes[2] = MPI_INT;
blen[3] = 2; byteOffSet[3] = (nbytes + floatsize + intsize); oldtypes[3] = MPI_INT;
blen[4] = ulen; byteOffSet[4] = (nbytes + floatsize + (intsize * 3)); oldtypes[4] = MPI_INT;
blen[5] = 1; byteOffSet[5] = sizeof(struct individual); oldtypes[5] = MPI_UB; >>



Ok, I'll start off by admitting that I'm just learning MPI and haven't used MPI_Type_struct.
But I do find a few thing curious in your code above.

1. You don't show where chromsize & nbytes come from. Are you sure they are correct?

2. You use sizeof(float) in byteoffset[2] which corrosponds to the second member of your struct which is a double. float & double are not generally the same size.

3. I'm not quitre sure what you mean by this statement: "I did deep copies of everything too make sure that my unsigned * and int * are equivilant to unsigned[] and int[]."
I assume you mean that you copied the contents of the vector, and not just the pointer to it, but the notation you used (make sure that my unsigned * and int * are equivilant to unsigned[] and int[]) is strange. In any case, I'd have a closer look at that code.

<snip>

 

[Locutus4657]

Well, you did catch at least one small error in my code... Yeah, I did mean too say "sizeof(double)" insted of "sizeof(float)"... I know that nbytes and chromsize are correct... I asked the same question in comp.parallel.mpi, and I got the bad news that I half expected... I can't give MPI_Type_struct the chrom * pointer because it won't point at the data like an array pointer will dispite the deep copy (yes, by deep copy I mean I malloced chrom and used a for loop too copy each index), apparently chrom * would point back to it's self, and then I would get nbytes of garbage after that... So I guess I would have to give MPI_Type_struct the absolute address of where everything in the struct is... Which of course BITES because this means I can't derive one general data type for an array of individauls (by the way, yep I'm having fun with GA's!) What I would have to do is derive a data type for each individual I'm about to transmit, and that of course bites... I guess the best(ish) way I can transmit the individuals is too do it the way I currently have it working, just transmit every member of individual individually... Oh well... So much for efficiantcy... Perhaps one of these days the kind folks setting the MPI standard will learn all about data abstraction and how to apply it too their library...



<<

<< I am trying to send the following structure using MPI_Type_struct:

struct individual
{
unsigned *chrom;
double fitness;
int xsite;
int parent[2];
int *utility;
}; >>



mmmm ... looks like fun with GAs?



<< My program crashes durring send, what happens is chrom transmits successfully, but everything else seems to be messed up. I did deep copies of everything too make sure that my unsigned * and int * are equivilant to unsigned[] and int[]. I know that the offset is supposed to be of type MPI_AINT, but I havn't seen any good description of how to use something of this type. If anyone in the forum is experianced with MPI and can help that would be great. I have my program working where I send each member of each individual I am transmitting individaully, but as you can imagin this is quite inefficiant.

derive my MPI type is cut and pasted below:

floatsize = sizeof(float);
intsize = sizeof(int);
blen[0] = chromsize; byteOffSet[0] = 0; oldtypes[0] = MPI_UNSIGNED;
blen[1] = 1; byteOffSet[1] = nbytes; oldtypes[1] = MPI_DOUBLE;
blen[2] = 1; byteOffSet[2] = (nbytes + floatsize); oldtypes[2] = MPI_INT;
blen[3] = 2; byteOffSet[3] = (nbytes + floatsize + intsize); oldtypes[3] = MPI_INT;
blen[4] = ulen; byteOffSet[4] = (nbytes + floatsize + (intsize * 3)); oldtypes[4] = MPI_INT;
blen[5] = 1; byteOffSet[5] = sizeof(struct individual); oldtypes[5] = MPI_UB; >>



Ok, I'll start off by admitting that I'm just learning MPI and haven't used MPI_Type_struct.
But I do find a few thing curious in your code above.

1. You don't show where chromsize & nbytes come from. Are you sure they are correct?

2. You use sizeof(float) in byteoffset[2] which corrosponds to the second member of your struct which is a double. float & double are not generally the same size.

3. I'm not quitre sure what you mean by this statement: "I did deep copies of everything too make sure that my unsigned * and int * are equivilant to unsigned[] and int[]."
I assume you mean that you copied the contents of the vector, and not just the pointer to it, but the notation you used (make sure that my unsigned * and int * are equivilant to unsigned[] and int[]) is strange. In any case, I'd have a closer look at that code.

<snip> >>

 

[Nothinman]

Well, you did catch at least one small error in my code... Yeah, I did mean too say "sizeof(double)" insted of "sizeof(float)"..

Little OT, similar things were mentioned on something else I read (not sure which forum, or maybe lkml heh), but you should use sizeof(var_name) instead of sizeof(var_type) because it allows for type-os like this and if you change the var type you only have to change it in one place.

 

[Armitage]

Ok, well here's a question. I assume you're implementing a parallel GA.
Why are you sending that whole structure? Why not just send the representation (chrom vector I assume) and an index. Then, after the eval, you send back the fitness value and the index and put them back together. Even if you're doing an island model, I don't see much point to sending the whole thing.

But then, I don't know what or why some of the things in your struct are there, so maybe you have a good reason!

FWIW, here's how you would send the whole thing with PVM (a much nicer library IMHO).
First I would make it a class instead of a struct. Then I would define member functions called pvm_pack() and pvm_unpack()

int individual:: Pvm_pack(unsigned chromsize, unsigned utilitysize)
{//I would probably make chromsize & utility size static members of the individual class
//Then you wouldn't have to send them each way
int err;

err = pvm_pkuint(&chromsize, 1, 1);
err += pvm_pkuint(chrom, chromsize, 1);
err += pvm_pkdouble(&fitness, 1, 1);
err += pvm_pkint(&xsite, 1, 1);
err += pvm_pkint(parent, 2, 1);
err += pvm_pkuint(&utilitysize, 1, 1);
err += pvm_pkint(utility, utilitysize, 1); //I don't know what utilitysize is from your post??

return err; //non-zero return indicates failure
}

int individual:: Pvm_unpack(void)
{
int err;
unsigned chromsize, utilitysize;

err = pvm_upkuint(&chromsize, 1, 1);
err += pvm_upkuint(chrom, chromsize, 1);
err += pvm_upkdouble(&fitness, 1, 1);
err += pvm_upkint(&xsite, 1, 1);
err += pvm_upkint(parent, 2, 1);
err += pvm_upkuint(&utilitysize, 1, 1);
err += pvm_upkint(utility, utilitysize, 1);

return err; //non-zero return indicates failure
}

Now, if I want to send a vector of individual objects called pop (for population), this is how I would do it:

pvm_initsend(PvmDataDefault);
err = pvm_pkuint(&popsize, 1, 1);
for(x=0; x<popsize; x++)
err += pop[x].pvm_pack(chromsize, utilitysize);

err += pvm_send(slv_id, msgtag); //you get slv_id when you spawn the slave process. you choose msg_tag. both are int

if(err)
///handle error...

Now to receive this message on the far end

int err;
unsigned popsize;
individual *pop

pvm_recv(pvm_parent(), msg_tag); //msg_tag is the same as what the message was sent with

err = pvm_upkint(&popsize, 1, 1);

pop = new individual[popsize];

for(x=0; x<popsize; x++)
err += pop[x].pvm_unpack()
if(err)
//handle error

In my opinion, this is much nicer then what I've seen in MPI so far.