Application Data Access – closing the circle
In the early years an application’s code was merely a reflection of the data it was meant to store and retrieve. For instance, a COBOL commarea from a program meant to access data on IBM IMS database had the structure that mapped to the data, on it’s various possibilities. Alongside with the CRUD methods implementation.
If you wanted to search through the data, you had to develop the code for it. That was achievable to the developers but consequently no Ad-hoc queries were possible for the end-users. Business intelligence was nothing but some pen and pencil with big paper stacks.
Then came relational databases. The Relational model, first proposed in 1970 by Edgar F. Codd, departed from this tradition by insisting that applications should search for data by content, rather than by following links. This was considered necessary to allow the content of the database to evolve without constant rewriting of applications.
Relational systems placed heavy demands on processing resources, and it was not until the mid 1980s that computing hardware became powerful enough to allow them to be widely deployed. By the early 1990s, however, relational systems were dominant for all large-scale data processing applications, and they remain dominant today.
The dominant database language is the standard SQL for the Relational model, which has influenced database languages also for other data models.
Finally to make my point about what this post is about, comes the trend of handling large data stores with NoSQL. It does not use SQL as its query language. NoSQL database systems rose alongside major Internet companies, such as Google, Amazon, and Facebook, which had significantly different challenges in dealing with huge quantities of data that the traditional RDBMS solutions could not cope with.
NoSQL database systems are developed to manage large volumes of data that do not necessarily follow a fixed schema. Data is partitioned among different machines (for performance reasons and size limitations) so JOIN operations are not usable and it differs from the early days IMS hierarchical database because ACID guarantees are not given.
Nevertheless, we’ve gone back to key/object systems with little flexibility, highly optimized for retrieve and append operations and often offer little functionality beyond record storage.
I state again that data development has gone full circle.
I like Internet Explorer!
It's pretty well written but It's nothing more than a designer rant and some trolling on IE.
It's true that IE falls way behind the competition on the browser extensions and integrated cloud service and it's a given fact that a lot of Windows users don't update their browser to the latest available version.
But IE9 is still a pretty good browser and the missing plug-in market is a positive thing when it comes to browser safety and stability.
Besides Internet Explorer at the work desktop, I also use Firefox and Chrome for personal stuff. Who of you have never seen this page on Chrome:
Manually placing pending OneNote updates to SkyDrive
On a careless fashion, I kept taking notes from tasks and e-mails and the pending updates stacked up. Final picture: ended up with a really stuffed set of notes missing from the online service.
No matter how I tried to start the process, the merge between the notes always failed on the Windows Live ID login step.
The web browser view on Skydrive presents the notebook as a single item, but OneNote stores the different sections of a notebook on separate “.one” files. For the whole notebook there is a table of contents “.onetoc2” file. So the potential solution was to replace the Skydrive files with the local backup files that had all the recent updates.
The solution wasn’t feasible on the Web browser Skydrive functionality, as it presents the notebook as a single item. It was a four step process:
- Get Skydrive to open on Windows Explorer.
- Backup your online notebook to a local folder and open it.
- Open the files from the local OneNote backup folder.
- Copy the local files over the old ones.
Backup your online copy to a local folder. Open the notebook on OneNote.
To get the local backup of your skydrive files, it should be on a folder path like “C:\Users\[Your user name]\AppData\Local\Microsoft\OneNote\14.0\Backup”. Replace [your user name] with the real deal. Open the notebook on OneNote.
With both versions of the notebook open, check if there are missing notes and copy/paste them into the copy with the least updates missing.
Finally, copy the whole “.one” files from the updated version, replacing the ones on Skydrive, using Windows Explorer.
HTH.
Dados.gov - transparência e a reutilização da informação do sector público
Citando o próprio site, o Dados.gov é um portal que tem como função publicar e agregar a informação produzida pela Administração Pública em formatos abertos que possam ser lidos e reutilizados por qualquer cidadão. O objectivo fundamental da criação deste portal visa facilitar o acesso à informação pública, promover a transparência da Administração Pública e ajudar na criação de serviços públicos electrónicos pela sociedade civil.
Já existe uma iniciativa global de transparência e reutilização dos dados que podem ser acompanhadas no portal OPEN! Government Data. A par da iniciativa portuguesa, já existem iniciativas relevantes, previamente lançadas por outros países:
O particular interesse da iniciativa portuguesa de dados abertos, é que propõe facilitar imenso a criação de um próprio ecossistema de satélites de informação, utilizando as suas origens de dados. Nesse sentido, disponibiliza:
- Um catálogo de dados: estão disponíveis todos os conjuntos de dados partilhados por instituições da Administração Pública
- Serviços de obtenção de Datasets através de protocolos REST com XML em padrão AtomPub (OData), JSON e outros.
- Tutoriais para programadores em .NET e Java.
Existe inclusivamente um inventário das aplicações já desenvolvidas por terceiros com base nas origens de dados do Dados.gov.
Este portal vai de certeza passar a ser a vítima preferida para alguns ensaios de programação.
Deixo um vídeo elucidativo dos projetos Open Government Data:
#opendata from Open Knowledge Foundation on Vimeo.
A implementação das boas práticas ITIL na Administração Pública: um estudo de caso
Há coisa de cinco anos, o Instituto onde trabalho submeteu-se a um processo de adopção do ITIL. Foi um trabalho estruturante de adopção de metodologias, criação de processos e auto-estudo, com uma prova de certificação para cada colaborador.
Numa pesquisa em nada relacionada com o tema, deparei-me com uma dissertação de mestrado, acerca desta adopção do ITIL por parte de uma entidade da Administração Pública, escrita pelo então diretor do departamento de sistemas.
Por ser um caso interessante onde a racionalização dos processos se traduz numa efetiva melhoria no serviço aos utilizadores e cidadãos, deixo aqui o endereço onde podem descarregar o documento:
A implementação das boas práticas ITIL na Administração Pública: um estudo de caso
Boas leituras!
Installing Sharepoint 2010 on server without Internet acess
The solution is to download the pre-requisites software and copy them to the server. Here's a list for future reference:
- Microsoft SQL Server 2008 Native Client
- Microsoft Sync Framework Runtime v1.0
- Windows Identity Foundation
- Microsoft Chart Controls for Microsoft .NET Framework 3.5
- Microsoft SQL Server 2008 Analysis Services ADOMD.NET
- Hotfix for Microsoft Windows - KB976462 (didn't need it, was already installed)
- Windows language packs
- Sharepoint language (Foundation and Server)
- Security Updates
For this and other Sharepoint 2010 issues I strongly recommend reading Professional SharePoint 2010 Administration
Requested registry access is not allowed
The origin of the error was the attempt the service made to write on the Event Log.
One of the changes we had made to the application was the application pool identity. It was running on Classic mode with a given domain account and we re-configured it to run on integrated mode with the application pool identity.
To grant rights to a given user account for writing on the Event Log, you should perform the steps to edit the registry described here:
- Find key "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Eventlog"
- Right-click and choose "Permissions".
- Put in the desired account
The catch is that the application pool is running with the Application Pool Identity Account. This is a Windows user account called "IIS APPPOOL\AppPoolName", which is created when the Application Pool is created, where AppPoolName is the name of the Application Pool.
On the permission dialog, search the local host for the "IIS APPPOOL\AppPoolName" replacing "AppPoolName" for your custom given name.
After that, the Web service worked fine.
Chrome browser slow on corporate network
Having it installed on the corporate office workstation, I noticed it was much slower and taking forever to load some really simple stuff.
After some research on Google, I found and tested successfully the following scenario:
- The proxy server is automatically configured through network policies.
- Configuring the proxy server directly, without automatic configuration, restores Chrome to its full speed.
- Start/Run the "Regedit" tool
- Find the key "HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings"
- Set the ProxyEnable entry to "1"
- Create a "ProxyServer" String entry and set it to «your proxy server address»:«port used» (ex: 10.0.211.11:8080)
- Finally, go to your browser's LAN settings and disable automatic configuration
How Google improves search algorithm
I enjoyed the video below because it makes evident that all the search enhancements made relies on the human factor. It's people who evaluate the accuracy of search results, the effective improvements achieved with potential algorithm updates, even help identify common misspellings that should be considered.
I agree that it's a scientific process and the Google search team really are scientists in the new ideas they have and the way they put them to test with real subjects: Humans.
A technology leader showing that huge amounts of data and vast infra-structure isn't enough to lead without people to sort it out:
LibreOffice spellcheck not working - try this
I couldn't find anything on the options manager that put spellcheck to work. Finally, inside the word processor, I selected "Tools > Languages > More Dictionaries Online":
It opened a browser window on a OpenOffice Wiki page. After searching a while, I landed on the following URL: http://extensions.services.openoffice.org/en/dictionaries
I downloaded the extension corresponding to my desired language, a .OXT file, opened with the LibreOffice extension manager and installed it.
I restarted the Word processor and it just worked. HTH!
Offline Google Mail without using a desktop client
For Gmail users, who use the Chrome Web browser, the use of a full desktop e-mail client is now optional.
Blind ambition
Their project is a device to help taking notes for students with vision disabilities.
The solution uses OneNote and is spotlighted in this inspiring short video:
Cloud Computing Data Center Architecture = Container
If the server stack of a service doesn't measure up to the performance standards required, scale with another container full of hardware.
Major players in cloud computing, Google and Microsoft are already defining their data center's architecture with this new order of greatness.
Google:
[youtube http://www.youtube.com/watch?v=zRwPSFpLX8I&w=560&h=349]
Microsoft:
All with a constant concern for energy-saving and the environment.
Tomboy sync using Windows and Ubuntu
If we're talking about utilities like Dropbox, it already supports synchronization clients for windows, Mac and Linux.
On this specific case, I wanted a decent note taking application that allowed me to sync between my Windows desktop at work and my Ubuntu laptop at home. A suggestion I found on the Web worked on file based software and Dropbox synchronizing those folders. That might work, but
The first step, if you don't have such an account, is to register on Canonical's Ubuntu One service. It gives you 5GB free storage to sync files, contacts and notes on all your Ubuntu machines. It's meant to support Windows OS soon enough.
The second step is to install Tomboy on the Windows machine. To install Tomboy on Windows, you need .NET framework and gtk-sharp installed before you can install Tomboy:
After installing Tomboy, start the application. It automatically sets up an icon on the system tray. Press the right mouse button on it and choose "Preferences".
Go to the "Synchronization" tab and select the "Tomboy Web" service. Ont the server insert the following address: https://one.ubuntu.com/notes/
Choose the interval between synchronizations. press "Save" and you're all done on the Windows box. For the Ubuntu machine, there is this detailed tutorial on notes configuration on the Ubuntu One wiki pages.
Distributed Transactions on Windows 2008 Systems – III
so far we've covered these aspects:
- Post I - How MSDTC works and relevant deployment scenarios and when do I need to perform additional configuration
- Post II - Defining a port range for MSDTC so it can be managed on a firewall configuration and enabling TCP/IP transaction communications.
Recalling the scenario for this post, we assume:
- static addressing of the hosts
- no naming servers available
- packet broadcast disabled
- firewall between application hosts and database hosts
On these conditions, MSDTC will not work because it cannot translate the NetBIOS names of the participating hosts into their IP addresses. Even without naming servers, DTC might try to resolve the hosts names with a network packet broadcast, but for the sake of this example we assume it's cut off by a firewall. Usually, even ICMP (ping) is cut off.
The remaining work around is to add address and name entries to a system file called "lmhosts". The "lmhosts" file is a local text file that maps Internet Protocol (IP) addresses to NetBIOS names of remote servers with which you want to communicate. It's similar to the regular "hosts" file but the latter one concerns general network lookup, like DNS replacement. The Lmhosts file is located in the "%SystemRoot%\System32\Drivers\Etc".
So, the first step is to edit the "lmhosts" file and add entries for each of the remote hosts involved. It's done adding a tab separated pair of IP address and NetBIOS name on each line. You can add the #PRE suffix to the lines so the host loads the lookups at boot time. For instance:
10.0.132.73 remote-mssql-host-1 #PRE 10.0.132.65 remote-mssql-host-2 #PRE
Afterwards, the TCP/IP connection properties need to be edited:
- On the Windows host, open the Network and Sharing Center. Press the "Local Area Connection" link.
- Press the "Properties" button. On the new window select the "Internet Protocol Version 4" and press "Properties".
- Press the "Advanced" button and select the WINS tab. Choose to enable LMHOSTS lookup and NetBIOS over TCP/IP.
Press "OK" on all windows. Perform these steps, together with the ones mentioned on the previous posts and try your luck.
Hope this helps.
Distributed Transactions on Windows 2008 Systems – II
The Windows Distributed Transaction Coordinator performs its communication using RPC communication, which by default means it requires 135 port and a set of dynamic ports above 1024.
A dynamic set of ports with no well-known range is not a best practice for network security. Given that, it's not a question of just asking the firewall administrator to open the default ports. We'll first configure a port range on the DTC service of the target machines.
On one of the hosts to configure:
- open the Windows Menu > Administrative Tools > Component Services;
- Drill-down to "My Computer", then select "Properties" from the mouse context menu;
- Select the "Default Protocols" tab. There should be a "Connection Oriented TCP/IP" item; Press the "Properties" button.
- Press the "Add" button and type in a suitable range like "50000-50200". This means DTC will use RPC thru ports 50000 to 50200.
Press OK on all windows and restart the server to assure it assumes the newly configured range.
Please note that this configuration has to be done on ALL hosts participating on the distributed transaction. If a given scenario has two separate MSSQL Server hosts and two more application servers, all four machines must have the port range configured on the DTC service configuration.
This step configured the port range for MSDTC on the target hosts. The next step is to request the firewall administrator to open the following ports between the two networks:
- 135: fixed standard port for RPC
- 50000-50200: the range we configured on the example. Could be another given range.
Besides the network firewalls, each hosts own Windows firewall configuration must be edited to allow the connections thru the newly configured port range. To do this, on each host, open the "Windows Firewall with Advanced Security" utility. If hints on that are needed, there's this article on Technet.
Talking about distributed transactions over the network, it's obvious TCP/IP communications ar involved. Consequently, this channel has to be enabled on the local DTC.
- open the Windows Menu > Administrative Tools > Component Services;
- Drill-down to “My Computer \ Distributed Transaction Coordinator \ Local DTC”, then select “Properties” from the mouse context menu;
- Select the “Security” tab. The following options need to be checked: Network DTC Access, Allow Remote Clients, Allow Remote Administration
- Still on the same window, on the Transaction Manager Communication: Allow Inbound, Allow Outbound, No Authentication Required
- Check the Enable XA Transactions, uncheck the SNA LU option if you're no using legacy systems.
Press OK on all windows. It should be all done. This scenario still relies on dynamically assigned IP addresses and windows network architecture with WINS servers.
On the next and final post of the series, I'll consider a worst case scenario:
- static addressing of the hosts
- no naming servers available
- packet broadcast disabled
- firewall between application hosts and database hosts
It should be relatively interesting since everything is configured manually and it's a not so uncommon scenario for highly secured, tightly managed networks.
Delete Blogger pictures from Picasa
Connecting VPN on Ubuntu
Given the possibility of connecting to my Workplace´s network through VPN, I faced the challenge of configuring a VPN network client on Ubuntu Linux (9.10). Our communications staff had a ready to use setup for Windows, but that didn't help much. So I took the exported Cisco PCF file and got engaged on this task.
Overall, it wasn't as tricky as I would expect. I performed the following steps:
- Open the Synaptic Package Manager
- Selected the following packages: vpnc, network-manager-vpnc, KVpnc
- Apply the changes
Having finished the installation:
- Go to the Application menu, select Internet/KVpnc
- Start the new connection wizard and import the pcf file
- Save your new profile
- Connect and ... that's it!
I tried to import the pcf file directly, without using the wizard and it didn't work. Anyway, when your VPN connection is active, open a terminal window and run the command "ifconfig". Your VPN tunnel should be listed on the active connections.
Network adapter disappears on Windows 7
I found out that, whenever I booted up the machine before plugging the network cable in, my network adapter vanished from the notification area as well as the Network and Sharing Center. A reboot, with the cable already plugged in, solved this little annoyance.
But, sometimes, the problem has it source between the computer and the chair. I was not aware of a nice feature on Power Management that allows the computer to turn off a device to save power.
So I just had to take the check off the turning off option and the situation ceased to occur. Maybe someday I'll put it back on ;)