Eleventh hour Phase III Q&A

Update:
Got this email:

Hi Professor,


Will it be possible to extend the deadline for submission of group project from 6 PM to midnight i.e. 12 AM.

There are couple of reasons behind this request :

1) We followed the instructions for extension of SPSS recommended by IT services. However it involved Virtual Box creation which is a terribly slow way to operate heavy data on SPSS (each excel file is 3-4 MB, and it takes 10 minutes just to transfer data from excel to SPSS run on Virtual Box)

2) A lot of time was spent looking for secondary data which we did not expect.

3) Conflicting deadlines for Pricing, CSOT, ENDM and BVFS between today and tomorrow.

Will be grateful if the extension is allowed.
Regards,
G 

My response:

Hi G and Team,


I have no problem with extending the deadline another 6 hrs.

I don't know how to program turnitin.com.

Chandana has set the deadline for 6 PM on the dropbox.

Let me try to reach her and see if she can extend this to 12 midnight.

Shall inform the class soon regarding this.

Sudhir

Hope that clarifies. Shall let you folks know soon via (yet another) mass e-mail..

Sudhir
------------------------------------------------
Hi all,

Got this series of mails from "t" just now. Have edited to remove specifics and focus on general take-aways to share with the class.

Hi Prof,
One question on cluster analysis-
While choosing variables for cluster analysis, can we pick 3 factor scores which are essentially 3 buckets of multiple variables and two individual variables (ie scores on ‘I am generally budget conscious’ and ‘ I carefully plan my finances’)? Does that distort the output? Can it be interpreted?
Regards,
T

My response:

Hi T,
No, that's perfectly fine. In fact, it is recommended when certain variables don;t load very well onto the factor solution in the upstream stage.
As for interpretation, yes it follows the same as it would if they'd been factors. Factor scores are now essentially variable values where we are concerned in downstream analysis.

And then T replied:

Thanks for that prof. The only problem is that the variables under question are becoming disproportionately important in the 'predictor importance' scores and are pulling down the importance of other factor scores.

At which I wrote:

Make sure all the input variables into a cluster analysis program are standardized (i.e. subtract mean and divide by std dev) before cluster analysis to remove variable scaling effects on clustering. That should help.

Sudhir

P.S.
Will keep you posted as more Q&A happens.

Phase III - Grading Criteria

Hi All,

Might as well outline some thoughts on the grading criteria for the project. These are indicative only and are not exhaustive. However, they give a fairly good idea of what you can expect. Pls ensure your deliverable doesn't lack substance in these broad areas.

1. Quality of the D.P.(s) - How well it aligns with and addresses the business problem vaguely outlined in the project scope document; How well can it be resolved given the data at hand. Etc.

2. Quality of the R.O.s - How well defined and specific the R.O.s are in general; How well the R.O.s cover and address the D.P.s; How well they map onto specific analysis tools; How well they lead to specific recomemndations made to the client in the end. Etc.

3. Quality and rigor of Data cleaning - The thinking that went into the data cleaning exercise; the logic behind the way you went about it; the ways adopted to minimize throwing out useful observations using imputations, for instance; the final size of the clean dataset that you ended up with for indepth analysis. The data section should contain these details, ideally.

4. Clarity, focus and purpose in the Methodology -  Flows from the D.P. and the R.O.s. Why you chose this particular series of analysis steps in your methodology and not some alternative. The methodlogy section would be a subset of a full fledged research design, essentially. The emphasis should be on simplicity, brevity and logical flow.

5. Quality of Assumptions made - Assumptions should be reasonable and clearly stated in different steps. Was there opportunity for any validation of assumptions downstream, any reality checks done to see if things are fine?

6. Quality of results obtained - the actual analysis performed and the results obtained. What problems were encountered and how did you circumvent them. How useful are the results? If they're not very useful, how did you transform them post-analysis into something more relevant and useable.

7. Quality of insight obtained, recommendations made - How all that you did so far is finally integrated into a coherent whole to yield data-backed recommendations that are clear, actionable, specific to the problem at hand and likely to significantly impact the decisions downstream. How well the original D.P. is now 'resolved'.

8. Quality of learnings noted - Post-facto, what generic learnings and take-aways from the project emerged. More specifically, "what would you do differently in questionnaire design, in data collection and in data analysis to get a better outcome?".

9. Completeness of submission - Was sufficient info provided to track back what you actually did, if required - preferably in the main slides, else in the appendices? For instances, were Q no.s provided for the inputs to a factor analysis or cluster analysis exercise?  Were links to appendix tables present in the main slides? Etc.

10. Creativity, story and flow - Was the submission reader-friendly? Does a 'story' come through in an interconnection between one slide and the next? Were important points highlighted, cluttered slides animated in sequence, callouts and other tools used to emphasize important points in particular slides and so on.

OK. Thats quite a lot already, I guess.

Sudhir

Phase III - D.P. definition travails

Update:
Hi all,

Another Q I got asked recently which, I think, bears wider dissemination.

1. The 40 slide limit is the upper bound. Feel free to have a lower #slides in your deliverable. No problem.

2. The amount of work Phase III may take, by my estimate would be about 15-20 person hours in all. Say about 3 hours per group member. Anything much more than that and perhaps you are going about it the wrong way. The project is the high point of applied learning in MKTR_125. So yes, 3-4 hours of effort on the project is not an unfair amount of load. Besides, effort correlates well with learning, in my experience.

3. Do allow yourself to have fun with the project - its not meant to be some sooper-serious burden, oh no. Keeping a light disposition, a witty touch, a sense of optimism and the big-picture in mind helps with flexibility, creativity, out-of-box and all those nice things, in my experience. Whats more, if you enjoyed doing the project, rest assured it *will* show in the output.

Hopefully that allays some concerns about workload expectations etc. relating to Phase III.

Sudhir

------------------------------------------------------------------------------
Folks,

The example PPTs putup don't have a D.P. explicitly written down because I had given the D.P. in project 2009. In Project 2011 however, you have been given the flexibility to come up with a D.P. of your own, consistent with the project scope. This is both a challenge and an opportunity.

I got some Qs on whether the handset section responses can be ignored because client is primarily a service provider. Well, the handset section contains important information about emerging trends in the consumer's mindspace, the mobile application space and the mobile-related brands' perception space. A telecom service provider would dearly like to know how many target segment people have/are switching to smartphones, which apps they use most often so that the carrier can emphasize those apps more and so on. So yes, the handset section should not be ignored, IMHO.

Hope that clarified.

Sudhir

--------------------------------------------------------------
Got an email outlining whether such-and-such was a good D.P. - R.O. combo.

Now, I can;t share what the D.P. itself was but my response to that team had generalities which might do with some dissemination within the class.

Hi R and team,


Looks good. However, I don't quite see what the decision problem is. Step 1 seems to state more an R.O. than a D.P.

Sure, sometimes a D.P. maps exactly onto a single R.O. and that may well be the case here. But such a D.P. would perhaps be overly narrow considering the project scope initially outlined.

I'd rather you state a broader D.P. and break it down into 3 R.O.s each corresponding to the S, T and P parts of S-T-P.

Well, that's my opinion, you don't necessarily have to buy into it. What you do is ultimately your call.

Do email with queries as they arise.

Good luck with the project and happy learning.
Sudhir

More miscell Q&A:

Dear Sir,


While running the factor analysis on the pychographic questions, do we re-label the levels of agreement and disagreement as 1-5? Will this help us in any way?

Another way out could be to label the responses as 1 and 0 where 1 is for a level of agreement and 0 for disagreement. In the Ice-Cream Survey HW the responses to these psychographic questions were binary which actually helped in the analysis.

A quick response will be really helpful. Our approach is to first segment the consumers using factor analysis and then find some interesting insights into usage based on these segments.

Cheers,

S

My response:

Hi S,
1-5 (or 1-7 in case of a 7 point scale) is the conventional practice. Safe to go with conventional practice.

You might as well choose to go with something else but may need to justify why.

Alternately, copy the data columns of interest onto a new sheet, and reduce the #responses to three (-1 for disagree, 0 for neither and +1 for agree), re-run factor analysis and see if the variance explained, factor structure etc that you now get is better than with 1-5.

Idea is that as and when you approach specific problems, you think up of neat, creative ways to negotiate it and move on. Therein lies the learning in the analysis portion for MKTR. :)

Hope that helped. Do write in with more queries should they arise.

----------------------------------------------------------------------------
Dear Sir,

We are facing an issue with regard to clubbing the dataset with Q48 (in terms of Rank) into the final data set that was uploaded earlier. The following are the discrepancies:

• The new Data set for Q48 does not have serial no’s to link it with earlier set

• The no of rows (entries) in the new data set for Q48 are more than the original one

• There should be ranks from 1 to 3 but we see that some data entries up to rank 9

It would be beneficial for all the groups to have a final data set with Q48(in terms of ranks) extracted from Qualtrics.

Looking forward for a quick response from your end.

Regards,

M

My response:

Hi M,

The respondent ID that is there (leftmost column in the Q48 dataset) can be used to match (via VLOOKUP func in excel) with the same rows in the original dataset. The blogpost made a mention of this specifically.

Once you match the Q48 dataset entries to the original dataset ones, the problem regarding #responses less or more in Q48 dataset also goes away.

The ranks 1 to 3 alone are relevant. Some people ranked more than the top 3 items, so you may get upto 9 but that we can ignore, if need be.

Hope that clarifies.

Sudhir
----------------------------------------------------------------
Received this email today fro group "J":

Dear Sir,

We have the following analysis for our MKTR project:

Decision problem: Foreign handset player wants to enter the Indian market
Research Object: Wants to identify the most lucrative segment to enter.

This would make some of the questions about service provision irrelevant. Do you think we are on the right track?
Regards,

My response:

Hi Team J,


The project scope was designed from the viewpoint of a large Indian telecom service provider.
So a purely handset-maker's perspective would be limiting, I feel and perhaps not entirely consistent with the original project scope document.

My suggestion is you consider modifying the D.P. such that the data on service provider characteristics can also be integrated and used in some manner. For instance, a handset maker looking to ally with a service provider, perhaps.

In general, the D.P. should not be so limiting as to banish a good part of the data we have collected from analysis. There is plenty of scope to creatively come up with D.P.s that while focussing on the handset side of the story also use telecom carrier data.

Hope that clarifies.

Sudhir

Project Phase III Q&A

Update:
A more recent email with similar Qs:

Hi Professor,

Hope you’re well!

I’m writing to ask some help with the data cleaning of the Project’s Phase III data. Can you please help with the following queries?

1. What does it mean when the data field has both blanks as well as “-99” as the response? Thought it was the same thing, yet Q45, for instance, has both fields.

2. Can you please also advise on how to handle coding of ranking/rating questions? Specifically:
a. Q41 – Rating of top two categories of channels watched
b. Q48 – Ranking of only Top 3 brands out of a list of about 10

3. Also, how do we code blank responses (especially for interval or ratio scaled questions)?
a. Do we put a “0” or leave it blank?
b. How does SPSS handle zeroes vs. blanks?

Please advise.

My response:

Hi A and Team,
1. "-99" means respondent has seen the Q but chosen to ignore it. Blank means the respondent never saw the Q, i.e. the skip logic didn't lead him/her to the Q in the first place.


2. Q41 - select multiple options is what was done for the tv channel Q. So, folks have selected 2 (and some have selected more than 2). There is no ranking implicit in what was chosen.

Q48 - I'll download this Q afresh so that the rankings are visible. At present, under 'download as labels', we are unable to see the rankings.

3. a. Depends on how many blanks are there. If the entire row is blank, drop that row. The Q was not relevant to the respondent and hence qualtrics skipped those Qs for him/her. If a few columns here and there are at "-99", then either impite mean/median for that column or "0" for "do not know". Its a call you have to take given the context the Q arises in.

b. Doesn't handle them very well but allows you to do some basic ops. SPSS will ask whether you want to exclude cases (i.e. rows) with missing observations or whether you would rather replace the missing cells with the column means. Choose wisely and proceed.

Am sending the Q48 ranking data afresh to the AAs for an LMS upload. Should be up for viewing and download soon. Use the respondent ID to vlookup and match like rows in the master dataset you are currently working on.

Hope that clarified things somewhat.
Sudhir

-------------------------------------------------------------------------------------------------
Got this email from a team:

Dear Chandana,
Phase 3 project requires a 40 slide PPT as our deliverable. I would like to get the following things clarified:
a)      Is 40 the minimum or maximum limit. It seems too big a task.
b)      Can we generate output tables of tools such as Cluster and Factor analysis and include it as per the content of the PPT or is it that the tables be part of the appendix alone.
c)       Is secondary data analysis mandatory/allowed/is optional. Since the entire survey cannot cover a limit of 40 slides through a primary survey alone.
Kindly help in getting these points clarified as we are in the process of finalizing our approach.

My response:

Hi Team G,
1. The 40 slide limit is the upper limit. Feel free to have your final PPT deliverable less than 40 slides long.
2. There is little point in pasting SPSS output tables for factor.cluster analyses inside the 40 slide limit, IMHO. I'd rather groups present the interpretation/info/insight that emerges from such techniques. A hyperlink to the appendices that contain the SPSS tables shouldn't hurt at all, though.
3. Secondary data usage is welcome as long as the sources are documented and cited meticuluously.
IMHO, the main challenges arise in deciding upon a suitable D.P. and its constituent R.O.s. What follows is straightforward once these are set. I'd say, don't be overly ambitious in defining your D.P. nor overly shallow in scope either.
I hope that clarified things at least somewhat. Pls feel free to write in with queries as and when.

Regards,

Sudhir

Data cleaning tips for Project Phase III

Hi all,

As you by now well know, one of the exam Questions related to output from your phase III project (the factor analysis one). While analyzing data for this particular problem, I did notice quite a few irregulartities with the data. This is typical. And hence, the need for a reasonable cleaning of the data prior to analysis.

For instance in the Qs related measuring to importance of telecom service provider attributes, there were a few pranksters who rated everything "Never heard of it." Clearly, its stretch to think internet savvy folks have never heard of even one of a telecom carrier's service attributes. Such rows should be cleaned (i.e. either removed from analysis or if there are gaps etc, then imputing missing values for these gaps etc) before analysis can proceed.

Some advice for speeding up the cleaning process:

1. Do *not* attempt to clean data in all the columns. That is way too much. Only for important Qs, on which you will do quantitaive analysis using advanced tools (factor/cluster analysis, MDS, regressions etc) should you consider cleaning the data. So choose carefully the columns to clean data in. Needless to say this will depend on what decision problems you have identified for resolution.

2.  An easy way to start the cleaning process is to check whether there is reasonable variation in responses for each respondent. Thus, for instance, after the psychographic Q columns, insert a new column and in it, for each row, compute the standard deviation of the responses of the psychographic Qs. The respondents with very high or low standard deviations should be investigated for possible data cleaning. Thus for example, this method would catch people who mark the same response for every Q. Or those who mark only extremes for some reason.

3. If there are gaps or missing values (these are depicted with -99 in the dataset) in critical columns, then you may consider replacing these with the median or the mean of that column. A fancier, more general name for such work is imputation which includes also various model based predictions to use as the replacement value. Without imputation, one will be forced to drop the entire row for want of a few missing values here and there. Always ensure, the imputed values are 'reasonable' before going down this path.

More apropriately, imputation might work better if used segment-wise. Suppose you've segmented the consumer base based on some clustering variables. Now since those segments are assumed homogenous in some sense, one can better impute missing values with segment means/medians perhaps.

4. Don't spend too much time on data cleaning also. In the first (half-)hour odd spent on data cleaning, chances are you will cover the majority of troubled rows. After that there is a diminishing returns pattern. So draw a line somewhere, stop and start analysis from that point on.

Update:
I'll share my answers here on the blog to any Qs asked to me by groups. Conversely, I request groups to first check the blog for whether their Qs have already been answered here insome form or the other.

Hope that clarifies.

Sudhir

P.S.
Any feedback on the exam, project or any other course feature etc is welcome. As always.

Homework and Exam related Q&A

Post-Exam Update:
I later noticed, after the exam had started that there was a typo in the binary logit question. The coeff of income^2 was shown as -.088 instead of -0.88. Hence, the predicted probabilities of channel watched for all 3 cases (min, max and mean profile of respondents) would now come up as 1. Folks who show the expression and calculations will get full credit for this problem.

A related Q regarding the DTH caselet. There are many possible assumptions you could make and from each, a different research design might flow. As long as you've stated clearly your assumptions and the research design that follows is logically consistent with your assumptions, you are OK.

Hope that clarifies.

Sudhir

Update:

Another thing I might as well clarify: - regarding the quant portion - only interpretation of a model and its associated tables will be asked for.

Even there, only those tables that we explicitly have discussed in class, not merely shown but discussed, will be important from an exam viewpoint.

Sudhir

Am getting quite a few Qs regarding this. A wrote in just now:

Hi Chandana,

Need a quick clarification. For this home work questions 1,2,3 and 4 are mandatory and 5,6,7 are optional. Is that right ?
Regards,
A

My response:
Sudhir
----------------------------------------------------------------
Got this just now:

Professor,

When I mailed you today morning, I had high aspirations of finishing my studies and then meeting you for a quick review. Unfortunately, I just finished going through Session 4 hand out. My sincere apologies but I guess I will have to cancel this appointment…and give the slot to a better prepared student…!!

Thanks,
K

My response:

That's OK. Drop by anyway. I expected I'd be busy during these office hrs but am (pleasantly) surprised. Seems folks have on average understood the material well and don't need additional office hrs now.

Besides, anyway, the exam is not designed to be troublesome. I wouldn't worry overmuch if I were you.

Sudhir

----------------------------------------------------------------


Hi all,

Was asked yesterday about this and might as well share with the whole class.

Q was that in the factor analysis homework, what to do if the factor solution with eigenvals>1 is still showing cumulative variance explained of <60%?

Well, if the % is in the mid to late 50s, just go with it.

If not, it would seem like the factor solution is not doing a great job of explaining variance in the input variables. This is presumably because at least some variables are not well correlated with the others and are hence weakening the factor solution.

To ID such variables, look either at the correlation tables or at the communlaities table. The variables that show least correlation with others should progressively be removed and factor anbalysis re-conducted at each step till the 60% criterion is met.

If a variable loads entirely on its own factor, drop that factor and use that variable as-is in downstream analysis.

Hope that clarifies. Any more such Qs and I shall share it here.

Sudhir

Hi A,

Yes, only Q1-4 are mandatory. Rest are optional in session 9 HW.

Project related Q&A

Update:

I split up the old post which had both Q&A as well as deliverable format. The old post is now exclusively deliverable format and is available here:

http://marketing-yogi.blogspot.com/2011/11/more-phase-iii-project-q.html

A recent post on take-aways from past projects (particularly those in 2010) can be found here:

http://marketing-yogi.blogspot.com/2011/11/take-aways-from-past-projects.html
------------------------------------------------------------------------------

Hi all,

Got asked a few Qs, might as well putup answers here.

1."The decision problem is vague. Should we focus more on telecom services or handset features?"

My response: The scope document is vague for good reason. Provides enough wriggle room to variously interpret decision problems. Pls come up with your own decision problem that is not-inconsistent with the project scope.

Regarding telecom service versus handset, well, the scope doc says the client XYZ is a major player in the telecom services space - so it is primarily a service provider. The handset features thing is likely secondary to its main goal of improving its position among telecom service providers.

Pls come up with a suitable D.P. and appropriate downstream analysis. The main theme of interest will be, how much does your D.P. align with the scope document and how logical and consitent is the donwstream analysis with the stated D.P.

2. Scope says, XYZ wants to know "where the market is going". What does this mean?

Well, XYZ will want to know many things, sure, but only so many are knowable with the data at hand. My intention behind that part was, illuminate the current standing of different offerings first and then project a few years into the future. Make assumptions as necessary. There'll be little data to back up projections into the future, understandably so. Shall later expand on wht I mean by this point.

More Q&A as they come will be on this page as updates.

Sudhir
-----------------------------------------------------------

Update: More project related guidance

In general, there are a few Questions which the client is likely to find of interest and which I think each project team could consider going through.

(i) What is the turnover (i.e. rate of change or switching) among attractive customer segments in telecom carriers and handsets? [Hint: See how long one has been with a carrier/handset Q among others]. Is there a trend? Anything systematic that might indicate the market is moving towards or appears to be favoring a certain set of attributes more than others?

(ii) What kind of usage patterns, apps etc are the attractive segments (say in terms of purchasing power or WTP etc) moving towards? More voice? More data? More something? From here can flow recommendations to the client on which applications to focus on, which platforms to explore alliances with etc perhaps.

(iii) Perceptual maps of current reality - where attractive customer segments perceive current carriers and/or handsets to be. What are the critical dimensions or axes along which such percpetions have formed? What are the attractive gaps in positioning that may emerge? Which niche perhaps can a new or existing service lay claim to on the positioning map?

More Qs will be added as they occur.

Pls note that it is *not* necessary that these Qs be answered or recommendations made along their lines. Its just a guide for teams to think about incorporating into their current plan/roadmap. Incorporating these Qs makes sense only if they align with the D.P.s and the R.O.s you have chosen.

Sudhir