You are viewing a read-only archive of the Blogs.Harvard network. Learn more.

Django nonrel (noSQL) (app engine)

June 28, 2013 — JaZahn

I’ve been working with this for the last couple weeks. I have been doing this for a couple reasons.
1) I wanted to play with noSQL
2) I’m developing and deploying with the google app engine — the app engine does have a mySQL option (Cloud SQL), but they have no free version of this — so to do a GAE app (entirely (including data) in GAE), you have to use the google data store or pay for SQL.

Now doing the design in a key-value noSQL wasn’t too bad. I just had a design that had each object containing the ids of the other objects that were associated with it both ways. This was somewhat redundant she thinking about it in a relational way, but it made queries simple and fast. The overhead is making sure the software keeps the ids updated in all related objects I.e. if something is deleted.

That wasn’t so bad. It was a fun to design things differently than I’m used to. Then I fleshed out the models in django.

The issue I ran into was django-nonrel. This is the project that converts django to be able to deal with non relational DBs. This changes core django abstraction models and consists of apps that can be installed locally or globally. So I installed them locally because I don’t like the idea of altering the framework globally.

It did not work with the latest version of django.

So I read through the docs a little more carefully to discover the core team quit this project over a year ago. Because noSQL wasn’t right for their projects and Cloud SQL was now available for app engine. The guy who took it over isn’t keeping it up to date with django. He’s about a year behind. It’s working with django 1.3. Current version right now is 1.5.1. That is enough to throw errors.

If django is making changes in point releases that cause the django-nonrel to become out of date, either django isn’t doing a good job at maintaining backwards compatability or (much more likely) django-nonrel project isn’t doing a good job keeping away from hacky implementations that rely on very specific django references.

I don’t have the time to debug django-nonrel. It’s like 4 apps that go in your django project.

Additionally I worked out what the cost of Cloud SQL would be for development and I can deal with less than $2 a month. When it goes production I can worry about dealing with my teams extremely poor mysql management or get my group to pay for a more substantial Cloud SQL instance.

Posted in ATG, Flashcards. Tags: , , , , , , , . Comments Off on Django nonrel (noSQL) (app engine) »

Back in Time with PHP

December 16, 2012 — JaZahn

I recently had the opportunity to work on a PHP application that was written by a student in 2004. Before the time of PHP frameworks and separation of MVC, when applications had SQL and HTML in the same files. Each page was a form that submitted to itself. Everything contained in one file per page. Pretty crazy, right?

Crazy like awesome. I had so much fun working on features for this old application. It wasn’t just nostalgic, it was like comfort food. Easy and delicious. I’m not really sure how comfort food is “easy”, but just go with it. I was practically begging to do more work on it.

I can’t imagine myself creating a new application like this from scratch, but I’ve been thinking about the benefits. Anyone with just a tiny bit of PHP experience could pick it up and maintain it, there’s no technology ramp-up time. A project like this could be orphaned and passed around between anyone. It could be used to teach people basic PHP within the organization. Say, people who are so Java-centric they refuse to learn anything new or different.

It was a good opportunity to take a look at the past and see what we’ve lost by going forward with technology.

Posted in Design & Modeling, Development, PHP. Tags: , . Comments Off on Back in Time with PHP »

Github zip doesn’t include Submodules

October 18, 2012 — JaZahn

Seriously, what were they thinking. This completely invalidates the use of submodules. We want our code to be available without having to know git. That’s the whole reason they have the download a zip link. But not including all of the code means you have to unzip the code and then use git to get the submodules.

Submodules are great for code reuse. But if I can’t intend my project to be downloaded by people that just want to install it and not code anything while using submodules. This is horrible.

Posted in ATG, Git, Quizmo, Version Control. Tags: , , , , , . 3 Comments »

Giving up on Yii Oracle Clobs

July 24, 2012 — JaZahn

Started off yesterday with the intention of trying to implement functional Oracle Clobs in Yii similar to how I implemented it in CakePHP a couple years ago. As yesterday went on I kept busting through boundaries and was feeling great about my progress. But Then I hit the roadblock and I wasn’t able to continue. It’s not worth spending another couple days on when I have a deadline for a pilot by Fall. 4000 characters is enough for the pilot.

The result has been to alter the Yii COciSchema. Just changing the declaration of text to a varchar2(4000).

   public $columnTypes=array(
        'pk' => 'NUMBER(10) NOT NULL PRIMARY KEY',
        'string' => 'VARCHAR2(255)',
        //'text' => 'CLOB',
        'text' => 'VARCHAR2(4000)',
        'integer' => 'NUMBER(10)',
        'float' => 'NUMBER',
        'decimal' => 'NUMBER',
        'datetime' => 'TIMESTAMP',
        'timestamp' => 'TIMESTAMP',
        'time' => 'TIMESTAMP',
        'date' => 'DATE',
        'binary' => 'BLOB',
        'boolean' => 'NUMBER(1)',
		'money' => 'NUMBER(19,4)',
    );

The issue is Yii (PDO) doesn’t support LOBs well at all, so that declaration didn’t make any sense to begin with.

Posted in ATG, Databases, Oracle, PHP, Quizmo, Yii. Tags: , , , , , . 1 Comment »

Yii default ORDER BY

July 19, 2012 — JaZahn

I ran into an issue with a minor difference between Oracle and MySQL. Apparently MySQL is better about returning rows in the order they were inserted than Oracle. Now if you want to let me know it’s wrong to assume results are returned in any specific order, I know! Neither gives any guarantee on the order of results without an ORDER BY, but Oracle is semi-random.

It took me a little while to find the right way to add default items to queries. Yii uses CActiveRecords for model queries. I already had an overridden CActiveRecord class from Yii handling “getLastInsertId” with Oracle. So I knew I would be able to use that somehow.

I finally discovered scopes. It allows me to define a scope:

public function scopes()
{
    return array(
        'sort_order'=>array(
            'order'=>'SORT_ORDER ASC',
        ),
        'id_order'=>array(
            'order'=>'ID ASC',
        ),
    );
}

and then use it as such:

MyModel::model()->id_order()->FindAllByAttributes(array('SOMECOLUMN'=>1))

But still I didn’t find anything on default scopes. On a whim I did a grep -i “defaultscope” on the code and discovered it exists in the Yii framework. So I was able to piece together the following:

public function defaultScope()
{
    return array(
 	'order'=>'ID ASC'
    );
}

and bam. Add that to my QActiveRecord, and it’s golden.

Posted in ATG, Databases, MySQL, Oracle, PHP, Quizmo, Yii. Tags: , , , . Comments Off on Yii default ORDER BY »

Git Template Application Repository

June 20, 2012 — JaZahn

I titled this post what I wanted, but the result wasn’t quite so succinct.

What I wanted was a git repository that contained a template for future applications. I have done so much what I’m considering good work with Quizmo, I wanted to create a skeleton framework that I could fork into applications. So all of the components that I’ve created could be contained in one place and when I update them in Quizmo, they would likewise be updated in the template repository, and future applications would be able to easily pull down those same changes from the template repo. So I could really live the dream of rapid development RoR has been trying to sell for the last decade.

My first thought was to create a repo that had a dummy application in it and just forking it. Then I should be able to pass things back via the upstream and down with merges. It seems like such a simple operation. Unfortunately, that’s not how forking works. The problem is if you want to merge upstream, you have to merge everything. You can’t just merge the things that are in the upstream, there’s no mechanism for adding or committing one revision without committing every revision prior to that one.

So the only way to have different components connected via other repositories is to have multiple submodules. Everything needs to be contained in its own directory. There’s also fake submodules*. Which just relies on a lot more setup, I haven’t encountered problems with using submodules yet, so that isn’t an issue.

Instead of having the awesome application template repo I wanted, I’ve got a bit of a mish mash which currently looks like this:

The most important thing with this discovery of using a mish mash of submodules is that I should modularize my code. That is to say I need to have all related components in the same place. That seems obvious, but when going with the flow of yii, I put things where yii wanted them, not necessarily where they would be best served. Like twitter bootstrap, which got just strewn about all over the app.

The authentication abstraction I’m using is an IdentityFactory that chooses which xIdentity component to use which is an extension of UserIdentity but each individual xIdentity calls on its respective extension. So do the xIdentities go with the appropriate extension or with the IdentityFactory/UserIdentity? Asking the question makes me think it has to be with the extension.

Interesting links:

* Update: Upon working more with submodules, and paying attention to the pitfalls (outlined here) I’ve actually become very opposed to fake submodules (see above). Fake submodules relies on the person checking out your code to check everything out, and leaves the repository with no explicit link to the submodules. You basically just have to know. Which is fine for one guy working on one project, but is horribly irresponsible for a developer working for anyone other than themselves.

Posted in ATG, Git, Quizmo, Version Control, Yii. Tags: , , , , . Comments Off on Git Template Application Repository »

Abstracting schemas for MySQL and Oracle with Yii Migrations

May 13, 2012 — JaZahn

Yii migrations are an interesting way to keep track of your database changes.  I was originally looking for a tool that would abstract the schema SQL — with something like the Yii ActiveRecord — so I wouldn’t have to write duplicate code for Oracle and MySQL.  I mean, that’s kind of the point of going with the PDO abstraction.

So more than a few forum question/answers led me to Yii migration:

http://www.yiiframework.com/doc/guide/1.1/en/database.migration

I seriously have that page open 3 times in my tabs and I have read it top to bottom probably 20 times.  I keep going back to it over and over hoping it will have more information.  It’s just a little short on answers to all of my questions.

As I said, all I wanted was a way to abstract the schema, so the original premise of “migrations” was a little different than what I was looking for, but it being the only non-custom option, I went forward with it.  The point of migrations is to make sure the state of your database matches the revision of your code.  So it’s sort of like its own hokey versioning system.  As such, it doesn’t function as simply as I might like it.  It’s not easy to focus on one migration if it’s not the last migration.  Because the point of migrations is that you’re moving forward and editing a past migration would be like editing revision X without it effecting all of the future revisions.  So it makes sense for what it is, it’s just awkward.  For one the file structure HAS to include the timestamp.  And it has to include that timestamp at the front of the filename, so autocompleting these migration filenames is a pain in the ass.

One important thing they don’t talk about in the documentation is that the list of what’s been migrated is added automatically to your database via the “tbl_migrations” — and please remember the “s as it will insert it as lowercase, which could be annoying for Oracle users, though Yii seems to handle it “appropriately”.

The only major hurdle here is dealing with autoincrement. The way this was dealt with was I created an Autoincrement class that I plopped in the migrations directory:
protected/migrations/Autoincrement.php

This meant that I could just create the tables like so:

class m120402_194059_Quizes extends CDbMigration
{
	public function up()
	{

		$this->createTable('QUIZES', array(
			'ID' => 'pk',
			'COLLECTION_ID' => 'integer NOT NULL',
			'TITLE' => 'string NOT NULL',
			'DESCRIPTION' => 'string',
			'VISIBILITY' => "integer NOT NULL",
			'STATE' => 'string',
			'SHOW_FEEDBACK' => 'integer',
			'START_DATE' => 'datetime',
			'END_DATE' => 'datetime',
			'DATE_MODIFIED' => 'datetime',
			'DELETED' => "integer NOT NULL",
		));	

		Autoincrement::up('QUIZES', Yii::app()->db->driverName);


	}

	public function down()
	{

		Autoincrement::down('QUIZES', Yii::app()->db->driverName);

		$this->dropTable('QUIZES');

	}
Posted in ATG, Databases, MySQL, Oracle, PHP, Quizmo, Yii. Tags: , , , , , . Comments Off on Abstracting schemas for MySQL and Oracle with Yii Migrations »

Yii Migrations: Dealing with auto_increment for Oracle and MySQL

April 4, 2012 — JaZahn

Oracle deals with auto_increment by using sequences and triggers:

create sequence test_seq
start with 1
increment by 1
nomaxvalue;

create trigger test_trigger
before insert on test
for each row
begin
select test_seq.nextval into :new.id from dual;
end;
/

And MySQL is just an auto_increment keyword on the end of a column declaration:

CREATE TABLE animals (
id MEDIUMINT NOT NULL AUTO_INCREMENT,
name CHAR(30) NOT NULL,
PRIMARY KEY (id)
)
So I created a simple Autoincrement class:

/protected/migrations/Autoincrement.php

Which I included in the migration file for a schema.  This class will handle checking the driver and executing the appropriate sql.
Autoincrement::up('USERS', Yii::app()->db->driverName);
Autoincrement::down('USERS', Yii::app()->db->driverName);

What needs to be remembered here is to add a “sequenceName” to the model class, that is of the form “TABLENAME_SEQ”.

Posted in ATG, Databases, MySQL, Oracle, PHP, Quizmo, Yii. Tags: , , , , , , , . Comments Off on Yii Migrations: Dealing with auto_increment for Oracle and MySQL »
«
»